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ABSTRACT 



A study was made of procedures used by industry for allo- 
cating research and development investments. Areas of inquiry 
included levels and trends of R&D effort criteria upon which 
to base budget decisions, personnel selection criteria, appor- 
tionment between basic and applied research and development, 
lead time to results, extension funding criteria, and worth 
evaluation. 

Results show a wide range of criteria in use, with princi- 
pal differences between industries and between companies of 
different sizes. 

Recommendations are made for use of the study results, and 
for further study and eventual development of a comprehensive 
decision model. 
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I. INTRODUCTION 



Within recent history, the United States has enjoyed a continually 
growing Gross National Product (GNP, the total value of all goods and 
services produced in a year) . However, total research and development 
(R&D) in the United States decreased steadily over the ten year period 
ending in 1974 from approximately 3.0 percent to approximately 2.3 percent 
or $32 billion of the total GNP of $1,396.7 billion. This decrease was 
shown both by changes in the percent of total GNP dollars spent for R&D, 
and by the decline of the percentage of population of scientists and en- 
gineers working in R&D in the United States [National Science Foundation, 
1975] . 

From Fiscal Year 1966 until the Fiscal Year 1977 budget, the research 
and development funding within the Department of Defense, and correspond- 
ingly within the Department of the Navy, has decreased in terms of buying 
power as a result of decreasing annual congressional budget authorizations 
The decreasing trend in R&D has been accompanied by directed reductions 
in military and civilian manpower. The share of manpower reductions which 
the "R&D community" has had to accept has reduced the capacity for effort 
accomplishment. These effects have been reflected in increasing difficul- 
ty of justifying budget, and selecting individual projects or technology 
areas to be supported. A study which might contribute to development of 
improved or additional useful judgement and decision criteria was believed 
to be timely and potentially beneficial. 

Due to the prominence of the buying power of the Department of Defense 
its declining trend in research and development has been thought likely 
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to be the most significant factor affecting the overall national trend. 

As a prominent counter-effect, however, total expenditures for industrial 
research and development more than doubled from 1960 to 1974 to more than 
$22 billion in 1974. Eighty percent of all industrial R&D funds were 
spent by 100 companies engaged in five industries. Those companies spent 
one-half of the R&D funds for improving existing products, 35 percent for 
developing new products, and 15 percent for developing new processes 
[National Science Foundation, 1975] . Accordingly, successful, profit- 
motivated industry was considered as a proper area to seek decision cri- 
teria which may have aided in the success and growth of industrial R&D. 

Different sources vary in their predictions for the future trends in 
research and development. 

R&D spending has bottomed out and will turn upward in this cal- 
endar year. Many factors, including the molasses-fast decisiveness 
of Congress will hold off any rapid gains, but gains will come 
slowly at first, then with increasing speed. 

We see a $34 billion year in the research and development in- 
dustry. It f s not a big gain over last year f s figure, but it is a 
gain, and that ! s a start in the right direction [Jones , 19763- " 

"Continued inflation in 1976 may cause continued economic weak- 
ness, Battelle comments. Nevertheless, in terms of current dollars, 

R&D activity seems to have returned to the high rate of growth that 
prevailed before 1968. 

The proportionate composition of both funding and performance 
of R&D will change little, Battelle says . . . [Chemical and 

Engineering News, 5 January 1976j. n 

The proposed Federal funded R&D budget for Fiscal Year 1977 is $23.5 
billion, up about ten percent over Fiscal Year 1976. This increase will 

give a solid effort gain providing the estimated inflation rate of 6.5 

percent is not exceeded [Chemical and Engineering News, 26 January 1976]. 

In view of the foregoing predictions, the timeliness of a study of 
expenditures for R&D in industry was considered appropriate. 
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A. THE PROBLEM 



The allocation of limited funding and manpower to nationally promi- 
nent research and development efforts has become a highly visible, poli- 
tically sensitive procedure. Within the Navy, development of adequate 
justification to satisfy the myriad of decision review echelons, often 
has included a field or laboratory commander. Navy Headquarters, Depart- 
ment of Defense reviewers and Congressional committees. This has created 
a detailed and extensive major task for the office supporting the project. 
Once justified, the political prominence of a project then virtually 
assures availability of supporting resources. Many other important sup- 
porting types of R&D, equally necessary to overall progress, never reach 
the political prominence required to get the financial support needed to 
do the task correctly. Historically, within this latter group has been 
general support funding for the Navy "Research and Development Labora- 
tories" under the support of the Chief of Naval Material, and the several 
"Development, Test and Evaluation" facilities of the various Systems 
Commands and equivalent echelons. 

Development and maintenance of "centers of excellence", generalized 
technology areas, unique test and evaluation capabilities and facilities, 
and support of academic, industrial, and analogous independent research 
and development endeavors have been difficult to support. 

The Committee on Federal Laboratories Ql973^ , in a study made for the 
Federal Council for Science and Technology, attempted to formulate a 
method to measure the performance of Federal research and development 
laboratories. It was recognized in that study that some peer criticism, 
periodic project reviews, time and cost studies, benefit analyses, and 
other rough measures of the quality of effort aided management in 
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assessing the performance of R&D laboratories in many cases. However, 
the general conclusion was that "no general performance measure for 
Federal R&D can be recommended at this time." 

B. POSSIBLE BENEFITS TO BE GAINED FROM ALLOCATION CRITERIA 

Realistic, relatively objective and non-political proposed evaluation 
criteria for R&D should simplify and expedite the budget allocation pro- 
cedure and lead to more productive research and development. 

Improved understanding of the interrelationships of resources such as 
manpower and key equipment which are becoming progressively costly and 
scarce should encourage more effective distribution, sharing or alloca- 
tion of such resources. 

Quantifiable decision factors could be discovered and developed which 
would replace the subjectivity of peer evaluation as comparative judge- 
ment tools for allocation decisions. 

C. OBJECTIVE 

The objective of this study is to determine industrial research and 
development resource budgeting, allocation, selection and evaluation 
procedures for possible application within the Department of the Navy. 

D. SCOPE OF THIS THESIS 

This thesis surveys the characteristics of various research and 
development decision policies and criteria as used by industry. 
Characteristics examined included budget allocation level, budget trend, 
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most important project selection criteria, personnel and facility assign- 
ment criteria, research and development worth evaluation, and other 
factors and interrelationships. 

This thesis was limited to study of major industrial organizations as 
defined in the discussion of the determination of survey frame. It was 
recognized that an analogous survey of other categories of research and 
development performing organizations such as academic institutions, 
Federal laboratories, and non-profit (or not-for-profit) laboratories or 
corporations probably would be required to develop a comprehensive deci- 
sion model for Navy or other government applications. Observations or 
recommendations developed from this effort were limited accordingly. 



E. HYPOTHESES 



The approach to identifying industry research and development deci- 
sion policies required adoption of some basic hypotheses as follows: 

1. The trend of R&D effort levels in general is increasing at a net 
of ten percent of industrial corporations. 

2. More than fifty percent of industry R&D efforts funding is based 
on a percentage of some financial index of the corporation 
concerned . 

3. For two-thirds of sponsoring companies, less than five percent 
of basic research efforts yield financially profitable results 
within five years. 

4. For two-thirds of performing companies, the majority of research 
efforts yielding results warranting follow-on development of new 
products, did so within four years of initiation. 

5. Availability/evaluation of technological expertise is the major 
factor is selection of the activity or individual to pursue the 
research and development effort in more than 70 percent of 
specific efforts. 

6. Financially quantifiable benefit of R&D effort is the most 
important factor in sixty percent of industrial evaluations. 
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Hypothesis 1 is an attempt to find if there is any correlation be- 
tween the increase in the R&D national budgetary level predicted by most 
literature, and the industry observed trend. The remainder of the hy- 
potheses developed, hypotheses 2, 3, 4, 5 and 6, are considered funda- 
mental and potentially useful to Navy decision model development. 



II. METHODOLOGY 



The survey methodology was designed to encompass the following 
phases : 



Phase 


i. 


Personal Inter ivews 


Phase 


n. 


Literature Search 






1. Search Technique. 






2. Literature Review. 


Phase 


hi. 


Development of a Survey 



1. Determination of Topics. 

2. Design of the Questionnaire. 

3. Determination of Frame. 

4. Administration of the Instrument. 

A. PERSONAL INTERVIEWS 

Recognition of the need and possible benefits of the survey, or a 
sequence of surveys as initiated by this effort, resulted from the pre- 
sentation of a Navy R&D manager guest lecturer. Discussion with the 
lecturer, subsequent lecturers having other government or industry R&D 
management experience and responsibility, and the authors f experience 
emphasized the potential utility of a viable non-specific R&D effort 
decision or justification model. 

Additional interviews were conducted with selected R&D managers at 
Naval Systems Command office levels. 
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B . LITERATURE SEARCH 



1. Search Technique 

An initial literature search was conducted using the National 
Technical Information System and the Defense Logistics Studies Informa- 
tion Exchange data bank systems. Concurrent library research of period- 
icals and books was pursued at public, university, and defense extablish- 
ment libraries. The services of a commercial information research 
organization were engaged. After consultation, a repeat search of the 
National Technical Information System was conducted. This search was 
augmented by searches of the Engineering Index and the Inform data banks. 
These searches were conducted using the key word combinations and group- 
ings of Research/Development (both words in any combination) , Industrial 
(with) Research, Economic Budget, Cost, Civilian Investment, Cost (with) 
Analysis, and Research and Development, and the time frame of 1970 and 
subsequent publications. Abstracts thus obtained, cross-referencing of 
articles, references and bibliographies of the items acquired, and the 
facilities of the Library of Congress, the National Science Foundation 
and additional search of university libraries yielded a wide spectrum of 
related publications. The more pertinent of these publications are ref- 
erenced as applicable, and enumerated in the List of References and 
Bibliography. 

2 . Literature Review 

During the literature review, the interrelationships of various 
facets of research and development were clearly evident as expected. 

Thus, it was infeasible to separate literature as exclusively relative 
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to any particular hypothesis. Facets and key concepts of- articles and 
publications were synopsized to aid correlation, and cataloging was 
limited to use as a ready reference system and for bibliography 
development . 

Literature relating to the characteristica judged most adaptable to 
possible Navy use formed the basis for developing the hypotheses to be 
tested. This literature further contributed to the concurrent develop- 
ment of the test instrument. 

C. DEVELOPMENT OF A SURVEY INSTRUMENT 

1. Determination of Topics 

The literature review provided the authors with a good under- 
standing of the problems of selecting R&D projects and evaluating the re- 
sults of R&D effort. The literature also generated a desire for informa- 
tion on more recent attitudes of high level executives toward the policies 
and the extent of R&D effort at their companies. In order to obtain the 
needed information, a questionnaire containing twenty-seven questions was 
designed and entitled "Survey of Research and Development Management" 
(Appendix B) . The questionnaire was directed toward high level management 
personnel that were expected to have a good understanding of the R&D 
policies and procedures of their companies. 

The topic areas of investigation were: (1) R&D effort level, 

(2) R&D effort trends, (3) R&D budget formulation, (4) R&D budget break- 
down, (5) expectation of results from R&D, (6) R&D funding sources, (7) 

R&D project selection criteria, (8) staffing R&D sections, and (9) R&D 
evaluation. 
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2 . Design of the Questionnaire 



The questionnaire was divided into three sections. Section One, 
Background Information, concerned the demographic aspects of the respond- 
ent and the respondent's company. Section Two, Budgetary Considerations, 
related to the company's policies and procedures on R&D budget allocation, 
funding, and expected results. Section Three, General Management, 
covered R&D staffing, program selection, effort implementation, effort 
trends, and program evaluation. 

3 . Determination of Frame 

A survey of major business enterprises was made to attempt to 
discover management criteria which might be useful in developing a manage- 
ment model for Navy or other Federal systems. The major businesses group 
was selected in order that any techniques discovered would be useful to a 
business comparable in size to major segments or laboratories of the Navy. 
Forbes' 500s Ql976j and Fortune 500 £l976^ listings of American business 
and industry were used as a starting base for defining the survey frame. 

The Forbes' 500s analysis tabulated four business measurement 
factors, namely assets, sales, common market value and net profit. Most 
organizations listed among the top 500s were in more than one category. 

In consideration of the second hypothesis, which anticipated that most 
industries planned research and development budgeting as a percentage of 
some financial index, the most probable business measurement factor was 
a percent of sales volume. The sales 500 listing was judged most appli- 
cable for the desired survey. The Forbes' 500 sales listing was then 
expanded by including additional companies from the Fortune 500 
tabulation to compile the final survey mailing list. 
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4. Administration of the Instrument 



All outgoing mailings were addressed to the "Director of 
Research/Development . " The cover letter forwarding the questionnaire 
attempted to explain the nature and goal of the survey, to sincerely 
acknowledge the importance of contributions which the leaders of 
American industry and business would provide, and to assure all poten- 
tial respondents that the protection of anonymity would be provided in 
the data analysis. 



III. ANALYSIS OF RESULTS 



During the literature review, definitions of basic and applied re- 
search seemed not to differ significantly among the Department of 
Defense, the National Science Foundation, academic institutions and in- 
dustry. Some government or industrial activities are, however, on 
occasion organizationally segmented into development, testing, evalua- 
tion, laboratory, design and other categories. Accordingly, to aid in 
consistency in response and interpretation of this questionnaire, the 
following definitions were provided. 

BASIC RESEARCH includes original investigations for the advance- 
ment of scientific knowledge that do not have specific commercial 
objectives, although such investigations may be in the field of 
present or potential interest of the reporting company. 

APPLIED RESEARCH includes investigations directed to the dis- 
covery of new scientific knowledge that has specific commercial 
objectives with respect to products or processes . . . , applied 
research differs from . . . basic research chiefly in terms of the 
objectives of the reporting company. 

DEVELOPMENT includes technical activities of a nonroutine na- 
ture concerned with translating research findings or other scienti- 
fic knowledge into products or processes. Development does not 
include routine technical services or other activities excluded 
from research and development. 
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A. GENERAL CHARACTERISTICS 



1. Responses to Instrument 

The survey instrument was mailed to 597 commercial organizations. 
Twelve responses were returned incomplete. Some of the reasons for in- 
complete forms were: (1) conflict with company policy as preventing in- 

formation release, (2) R&D was conducted by subsidiary company, there- 
fore desired information was not available, and (3) the company does not 
conduct research or development. Completed questionnaires, with minimal 
if any missing data, were received from 147 respondents; these form the 
basis for the data analysis. Twenty-six percent of the companies 
responded, with ninety-three percent of responses analyzable. 

2 . Analytical Techniques 

Upon receipt, each completed questionnaire was reviewed, response 
to Question 4 of Section One which gave the position title of the res- 
pondent was coded, missing data points were noted, voluntary comments 
were tabulated, and data was prepared for computer analysis procedures. 
Computer analysis was conducted utilizing the Frequencies with Statis- 
tics, Crosstabulations, Histograms, Scattergrams and Non-Parametric 
Correlation subroutines of the Statistical Package for the Social 
Sciences [jtfie, 1975^]. 

Possible answers offered for selection for a specific question 
were progressive in value, but not necessarily uniform in value distri- 
bution. For example, Question 1 of Section One asked for the number of 
laboratories maintained by the company, giving a selection of answers as 
1, 2 to 5, 6 to 10, 11 to 15, or over 15 laboratories. While these 
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responses progressively increased in size or extent*, the amount of change 
from one possible answer to the next was not always uniform; thus, in 
statistical terms, the response ranking was ordinal (progressive), but 
not interval (in equal sized intervals). Kendall’s or Spearman’s corre- 
lation formulae were therefore appropriate for comparing data of this 
type. Application of these tests determined the relationship of factors 
(variables) tested. The magnitude of the value computed indicated the 
acceptance (similarity) or rejection (difference) of the related corre- 
lation or hypothesis. The Kendall’s or Spearman’s significance number 
(0.05, 0.01 or higher level of statistical significance, which signifies 
95 percent, 99 percent or higher confidence level respectively) indi- 
cates the probability that the relationship thus determined is not 
accidental . 

Coded answers to questionnaires were analyzed using the computer 
programs referenced above. The Frequencies subroutine was used to in- 
vestigate the number of times a specific answer was indicated for each 
question. The Histograms subroutine was used to plot the distribution 
of these answers. The computed Statistics analysis indicated statistical 
characteristics and percentages of answers as related to a particular 
question. Crosstabulations and Scattergrams analyses were used to in- 
vestigate possible correlations between specific factors (variables) 
which might yield clues to the manner in which industry relates these 
factors. The results of the Non-Parametric Correlation computer program 
(Kendall’s Tau and Spearman’s Rho) were used to determine the degree to 
which the relationships were reliable or accidental. 

The most illustrative computer output results were reproduced 
and included in the appropriate Appendix of this thesis. 
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Analysis of responses was organized in the following discussion 



in relation to specific questions, groups of questions, segments of the 
questionnaire, and as applicable to a specific hypothesis, in the manner 
considered best explanatory. 

B. SECTION ONE - BACKGROUND INFORMATION 

This group of questions was included to provide background informa- 
tion on the company of the respondent and to provide credibility to the 
information obtained from the survey. 

1. Number of Laboratories 

Question 1. How many research and/or development laboratories 
or facilities does your company operate? 

(1) one, (2) 2 to 5, (3) 6 to 10, (4) 11 to 15, 

(5) over 15 

The U. S. Navy has many R&D Laboratories over the United States; 
therefore, it was believed important to obtain information concerning 
R&D management from some industrial companies with multiple facilities 
that possibly would be comparable to U. S. Navy R&D operations. 

Approximately 42 percent of the companies responding had from 
two to five laboratories; 28 percent had only one. Eighty-six percent 
of all organizations had from one to ten laboratories. Only nine per- 
cent of the organizations, all with 10,000 or more employees, had more 
than 15 R&D facilities. 

2 . Company Size 

Question 2. Your company size as indicated by total number of 
employees : 

(1) Less than 5,000 (4) 20,000 to 39,999 

(2) 5,000 to 9,999 (5) 40,000 to 79,999 

(3) 10,000 to 19,999 (6) 80,000 or more 

20 



Companies with 10,000 or more employees consumed over four- 
fifths of U. S. industrial R&D funds in 1971. One hundred of these com- 
panies accounted for three-fourths of all industrial R&D for that year 
Qiorwitch and Prahalad, 1976J . The size of the respondents company was 
considered important, because the larger companies probably more closely 
approximate the U. S. Navy. 

The number and percentage distribution of various size respond- 
ing companies, as measured by their respective number of employees, is 
shown in Table 1. 



TABLE 1 

DISTRIBUTION OF COMPANIES BY SIZE 



Number of 
Employees 

under 5,000 

5.000 to 9,999 

10.000 to 19,999 

20.000 to 39,999 

40.000 to 79,999 
over 80,000 



Number of 

Responding Companies 
9 
25 
54 
29 
20 
10 



Percentage of 
Responding Companies 

6.1 

17.0 

36.7 

19.7 
13.6 

6.8 



All possible answers to Questions 1 and 2 received response, 
thus showing that all types of major industry were represented. 

There was a moderate positive correlation between the company 
size and the number of laboratories, as signified by a Kendall Tau of 
0.24 with a significance of .001. This confirms the expectation that, 
in general, a large number of facilities would be operated by a 
relatively large company. 
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3 . Industry Classification 



Question 3- As a very general industry classification, the 

greater portion of your company’s activities would 
be : 

(1) Durable goods industry 

(2) Non-durable goods industry 

(3) Services industry 

(4) Diversified industry 

This question was prompted by a study by Gellein and Newman 
[ 1973 ] on the accounting practices of several companies regarding costs 
incurred in industrial funded R&D. This study indicated approximately 
47 percent of the responding companies were engaged in durable goods in- 
dustries, 37 percent in non-durable goods, and 16 percent were diversi- 
fied. Services industries were not separately considered in this study. 

Durable goods were considered as items produced with an expected 
normal life of more than a year, such as machinery, automobiles and 
buildings. Non-durable goods were considered as being consumed or ex- 
pended within a relatively short period. Services referred to industries 
providing consumers with benefits other than physical goods, for example, 
airlines, utility companies and insurance companies. The diversified 
companies were engaged in some combination of the above industries. 

It was considered to be important to include the category of 
"services industry," because several services organizations are included 
in the Forbes’ 500s and the Fortune 500 lists. Also, several Navy facil- 
ities’ missions include consulting, design support, inspection, test and 
evaluation and other functions which might logically be classed as 
services . 

The categories of activities or products of the responding com- 
panies were widely distributed with 33 percent of the companies produc- 
ing durable goods, 31 percent producing non-durable goods, 23 percent 
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diversified in products and interests, and only 13 percent engaged in 
service oriented activities. 

Nineteen of the twenty companies with more than ten laboratories 
were engaged in the production of durable goods or diversified products. 
No service oriented company had more than five facilities, and 71 per- 
cent of them had only one laboratory. The services organizations were 
concentrated in the middle sized group. 

4 . Respondent Corporate Level 

Question 4. (Optional) Please specify your official title in 

the company organization. 

It was believed that the information obtained by the survey 
would have greater credibility if provided by a high level executive of 
the company who probably would have an optimal understanding of overall 
corporate policy. 

Four categories were defined and coded. Category one included 
presidents and vice-presidents, category two was defined as a major 
segment director or division manager, category three was related to a 
department or section manager, and category four was assigned to the 
respondents not identifying their respective title. Thirty-nine percent 
of the respondents who indicated their title were either presidents or 
vice-presidents, 38 percent were segment directors, 12 percent were de- 
partment or section managers, and only 11 percent remained anonymous. 

The very high percentages in the first two categories indicated a high 
degree of executive interest and responsiveness. This factor was con- 
sidered to infer substantial validity of the policy indications shown 
by the remaining data. 
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C. SECTION TWO - BUDGETARY CONSIDERATIONS 



This group of questions was designed to yield information on the re- 
cent industry R&D effort trend, the basis on which companies decide how 
much to spend on R&D, how the R&D effort is divided, lead times to reach 
R&D profitability, and what portion of R&D effort is company financed. 

1. R&D Effort Trend 

Question 1. How do you perceive your company’s R&D effort 

over the past five years? 

(1) No significant change 

(2) Increased significantly 

(3) Decreased significantly 

(4) Suspended R&D efforts 

Laserson and Sperling [ 1972 ] found from a survey of a large num- 
ber of industrial research organizations that 47 percent of responding 
companies had reduced their R&D effort during the period 1968 to 1971. 

An opinion poll by Industrial Research [^January 1975^ summarized 
the question "Is your company holding its R&D expenditures about even 
with those for 1970-73?” The results, when grouped, showed 30 percent 
about equal, 32 percent increasing, 28 percent decreasing, one percent 
dropped R&D, and eight percent other. This article further stated that 
”In 1973, 81% of the respondents felt that the amount of R&D in their 
company would increase, while in 1974, only 62% felt that their 
expenditures would remain equal or increase slightly.” 

The Federal budget for Fiscal Year 1977 submitted to Congress 
contained an eleven percent increase over Fiscal Year 1976 for Federal 
funded R&D. This increase of $2.5 billion brings the total Federal R&D 
budget to $22.6 billion ^Chemical and Engineering News, 2 February 19 76^. 

The intent of this question was to learn the overall trend 
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experienced by major industries (as recognized by Forbes and Fortune) in 
research and development effort, in terms of combined internally and ex- 
ternally funded effort, for comparison with the referenced studies. A 
general R&D growth pattern over the last five years was indicated based 
on a 46 percent response of no change of effort together with a 45 per- 
cent response which showed a marked increase. Only nine percent of re- 
spondents recorded a marked decrease, and none suspended research and 
development during this period. The first hypothesis of this thesis, 
based on recent literature, predicted an increasing trend at a net of 
ten percent of industrial corporations. "Net” was intended to mean the 
overall effect; that is, the percentage increasing minus the percentage 
decreasing. This survey strongly supported acceptance of the first 
hypothesis with a net of 36 percent indicating an increasing trend. 

Further correlation with answers to Section One questions re- 
vealed that of the 30 companies with more than 40,000 employees, 17 had 
stable R&D efforts, 11 had marked increases, and only two had marked de- 
creases. The above data inferred a much more optimistic future for 
growth of research and development effort than was reported in the 
studies of Laserson and Sperling, and the opinion poll of Industrial 
Research. 

2 . Percentage of Operating Budget 

Question 2. What percentage of your total operating budget does 

your R&D effort constitute? 

(1) Less than 2% (4) 8 to 10% 

(2) 2 to 4% (5) Greater than 10% 

(3) 5 to 7% 

The R&D expenditures of a company, as a percentage of its operat- 
ing budget, provides a measure of the n R&D intensity” level. Companies 
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are usually considered to be in the high intensity category if their R&D 
effort is more than two percent of the operating budget or of net sales 
^National Science Foundation, 1975^. 

Over 62 percent of the companies responding indicated that less 
than two percent of their total operating budget was devoted to research 
and development. By the National Science Foundation criterion, these 
were classifiable as not R&D intensive. The moderately intensive group 
of from two percent to four percent included 26 percent of the respondents. 

Only five companies indicated greater than ten percent commitment; 
the two largest of these companies classified themselves as durable goods 
industry, two others as diversified, and the fifth as non-durable goods 
industry. The highly intensive research and development effort was not 
restricted to any specific category or size of industry. 



3. Financial Index and Percentage of Index 



Question 3. Annual R&D budgetary level is planned as: 

(1) A percentage of net sales 

(2) A percentage of net profit before taxes 

(3) A percentage of net profit after taxes 

(4) A percentage of total operating budget 

(5) Specified other corporate financial index 

(6) Limited to active contracts; i.e., no in-house 
development 

(7) Not related to a financial index or factor of 

corporate operations. General basis is 

Question 4. Total R&D budget is limited to: 

(1) Not to exceed 2% of the above indicated 
financial index 

(2) Not to exceed 4% of the above indicated 
financial index 

(3) Not to exceed 7% of the above indicated 
financial index 

(4) Not to exceed 10% of the above indicated 
financial index 

(5) Other 

The National Science Foundation £l975[] definition of R&D 
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intensive effort as a percentage of operating budget or sales was 
reflected in these questions. 

The second hypothesis proposed that more than fifty percent of 
industry R&D efforts funding was based on some corporation financial in- 
dex. Questions 3 and 4 attempted to prove the hypothesis, and to deter- 
mine the extent of the budgeting based on the respective indexes. 

The first analysis of coded answers to Question 3 indicated 
that 61 percent of answering companies do not budget R&D based on any 
financial index. An additional four percent used active contracts as a 
basis for budgeting. Therefore, the second hypothesis was rejected. 

The null hypothesis, that less than fifty percent of industry R&D 
efforts funding is based on a percentage of some corporate financial 
index, was accepted. 

Of the 35 percent that did specify a financial index as a base 
for budgeting, nearly two-thirds (30 companies) used a percentage of 
sales . 

Sixty-one percent of the respondents also indicated, in Question 
4, that they used a basis other than a percentage for funds allocation. 
Nearly 13 percent did not answer this question. 

The last answer available for selection in each question was in- 
tended to allow respondents to indicate some financial index or charac- 
teristic not previously discussed in the literature or otherwise consi- 
dered. In this respect, all but one of the companies specifying n not 
related” to Question 3, also answered "other" or did not answer Question 
4. However, most did offer clarifying comments as tabulated in 
Appendix A. 

Of the 88 responses marked "not related to any index," 35 of 
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these, or 24 percent, indicated the R&D budgetary level was determined 
by the company management's assessment of needs and requirements. Seven 
respondents indicated R&D budget was planned as opportunities arose, and 
six indicated the value of a project was the determining factor. 
Responses tabulated in Table 2 show that the most frequently indicated 
answer was "need." 



TABLE 2 

INDUSTRIAL BASIS FOR R&D BUDGET PLANNING 

Basis or Index Company Responses Received 

Number of Responses Percentage of Responses 



Perceived Need 


35 


24 


Net Sales 


30 


21 


Total Budget 


12 


8 


Opportunities 


7 


5 


Value of Projects 


6 


4 


Outside Financed 


6 


4 


Profit After Taxes 


3 


2 


Other Index 


3 


2 


Profit Before Taxes 


2 


1 


Miscellaneous 


40 


28 


NOTE: Percentages are rounded 


to the nearest 


whole number. 



The respondents indicating "other" as an R&D budget limit, speci- 
fied various criteria for their maximum R&D budget level. The most fre- 
quent comment, approximately 20 percent, stated that their company had 
no specified limit on the R&D budget amount. Of the 90 responses that 
indicated "other" budget limit, 34 did not volunteer any criteria for 
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setting their R&D budget limit, thereby restricting* analysis . The 
specific comments on this question are also included in Appendix A. 

4. R&D Effort Distribution 



Question 5. What percentage of R & D effort is directed toward 
basic research? 

(1) Less than 5% (3) 10 to 14% (5) 20% or greater 

(2) 5 to 9% (4) 15 to 19% 

Question 6. What percentage of R & D effort is directed toward 
applied research? 

(1) Less than 10% (3) 20 to 29% (5) 40 to 49% 

(2) 10 to 19% (4) 30 to 39% (6) 50% or greater 

Question 7. What percentage of R & D effort is directed toward 
development pro j ects ? 

(1) Less than 20% (3) 40 to 59% (5) 80 to 100% 

(2) 20 to 39% (4) 60 to 79% 

These questions were designed to give information on the relative 
emphasis on industrial basic research, applied research, and development 
to compare with the literature findings. In an attempt to achieve uni- 
formity in answers and interpretation, National Science Foundation defi- 
nitions of basic research, applied research, and development were 
included in the questionnaire instructions and were repeated in part D 
above . 



The breakdown of overall resource allocations to research and 
development as reported in 1974 was 13 percent for basic research, 23 
percent for applied research, and 64 percent for development. This in- 
cluded efforts funded by Government, industry, educational institutions 
and others. The industrial funding allocation was three percent for 
basic, 18 percent for applied, and 79 percent for development ^National 
Science Foundation, 1975J. 

The Federal Government share of the basic research effort has 
increased from 60 percent to 70 percent over the time period from 1960 



29 



to 1974. The Universities have increased the funds • expended on basic 
research from six percent to 11 percent of the total research budget 
over the period from 1960 to 1974, while industrial funding has decreased 
for basic research from 28 percent to 11 percent over the same period. 

The trend by industry to shift their effort toward applied research and 
development ,! has been so great that, in some cases, there has been a 
danger of its squeezing out disinterested research, which would have 
been as absurd as killing the goose that lays the golden eggs ^Auger, 
1975] 

As much as 90 percent of an industrial company’s development 
effort is engineering related to new products and the remainder is re- 
search according to the vice-president of engine'ering of a large indus- 
trial company ^Commerce Today, 1974^. 

In contrast to some of the above literature indications, this 
survey seemed to indicate more emphasis on applied research, and a slight 
decrease in development effort. The order of importance was the same as 
previously reported for industry, however. Of the companies responding, 
80 percent allocated less than five percent of their R&D budget to basic 
research. Over 50 percent denoted that 20 percent of more of the R&D 
budget was allocated to applied research, and 60 percent directed 60 per- 
cent or more to development. The relative efforts by category of R&D, 
as indicated by survey responses, are shown in the Computer Output for 
basic research, applied research, and development. 

Table 3 relates these breakdown ranges and shows the shift in 
the nature of R&D effort as indicated by the responding companies. 
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TABLE 3 



RESEARCH AND DEVELOPMENT EFFORT EMPHASIS BY CATEGORY 



Percentage 

of 

Effort 



Under 


5 


5 


to 9 


1 


10 


to 


14 


15 


to 


19 


20 


to 


29 


30 


to 


39 


40 


to 


49 


50 


to 


59 


60 


to 


79 


80 


to 


100 



Percentage of Responding Companies 



Basic Research Applied Research Development 



80.1 



6.8 



4.8 



4.8 



* 3.4 



19.7 



24.5 



16.3 

10.2 

12.2 

** 17.0 



4.8 



7.5 



27.2 

25.2 
35 .4 



* 20 percent or greater. 
** 50 percent or greater. 



5 . R&D Lead Time 



Question 8. What percentage of basic research projects yield 

financially profitable results within five years of 
initial research? 

(1) Less than 2% (4) 8 to 10% 

(2) 2 to 4% (5) Greater than 10% 

(3) 5 to 7% 



Question 9. Of those basic and applied research efforts that 
lead to a decision to go ahead with development , 
the majority of them involve a time period from 
initiation to development decision of: 

(1) Less than 2 years (4) 11 to 20 years 

(2) 2 to 4 years (5) More than 20 years 

(3) 5 to 10 years 

Question 10. Of these R&D projects that lead to useful or 

profitable products, the majority of them involve 
a time period from initiation to profit of: 

(1) Less than 2 years (4) 11 to 20 years 

(2) 2 to 4 years (5) More than 20 years 

(3) 5 to 10 years 
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It is very difficult to determine an average for the percentage 
of projects that became successful out of all the projects that were 
begun in the R&D community. It has been estimated that among products 
that have emerged from R&D departments as technical successes, only one 
in three ultimately develops into a commercial success. Other estimates 
state that only one of twenty-five products that are test marketed 
become successful Qloman, 1968^ . 

M The average period elapsing between the time when a new process 
is discovered by applied research and the time when the substance or 
apparatus produced by the process is placed on the market has been great- 
ly reduced since the beginning of the century - from several years to a 
few months in some cases [^Auger, 1975J." 

The anticipated return on R&D takes place beginning three to ten 
years after project initiation into the development phase. It is not 
unusual for twenty years to elapse, from the basic research phase, before 
commercialization of a product begins j^Laserson and Sperling, 1972^. 

The cost of implementing an invention or discovery can create a 
long delay in gathering needed funds. The capital needed to apply an in- 
vention to a useful purpose averages about five times the cost of the re- 
search and development that made the invention possible [jhe Economist , 

1975]. 

It was evident that very few R&D projects which were begun ever 
became successful, and that those that were successful had long lead 
times. The above questions, as a group, attempted to determine the 
current success ratio and relative lead time of the successful projects 
or efforts. 

Of the companies responding, 68 percent indicated that less than 
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two percent of their basic R&D projects became profitable within five 
years after beginning the research. An additional three percent of the 
companies indicated that from two to four percent of their basic re- 
search was profitable within this time frame. The third hypothesis pro- 
posed that for two-thirds of sponsoring companies, less than five percent 
of basic research efforts yield financially profitable results within 
five years. This total of 71 percent with less than five percent prof- 
itability from basic research within five years was considered as vali- 
dating and accepting the third hypothesis. By contrast, 13 percent 
indicated that over ten percent of their basic R&D projects yielded a 
profit within five years. 

In response to Question 9, the largest group, 44.4 percent, in- 
dicated that the decision to go into a development program for a project 

occurred from two to four years after initial research began. In addi- 
tion, 21.8 percent indicated less than two years to a development deci- 
sion. The fourth hypothesis predicted that, for two- thirds of the 

performing companies, the majority of research efforts yielding results 
warranting follow-on development of new products did so within four 
years of initiation. When combined, these two groups represent 66.2 
percent of the total respondents; therefore, the fourth hypothesis is 
marginally rejected. The null hypothesis was accepted, that for one- 
third of performing companies, the majority of research efforts yielding 
results and warranting follow-on development of new products required 
more than four years from initiation. The longer period of five to ten 
years was indicated by 31.0 percent of the companies, and the period 11 
to 20 years by an additional nearly three percent. 

In response to Question 10, nearly 50 percent of the respondents 
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stated that the R&D projects that did become profitable, did so within 
five to ten years after initiation. Only 12 percent indicated lead 
times less than two years, and 32 percent realized profits between two 
and four years subsequent to initial research. Recognizing that a deci- 
sion to go ahead with development of new products (Question 9) does not 
automatically mean profitability, Question 10 attempted to determine any 
relationship between the time to "go-ahead 11 and the time to profitabil- 
ity. The computed Kendall *s Tau correlation factor related to these two 
variables was 0.43 with a significance of 0.001, which inferred a very 
high confidence in a strong correlation. This correlation means that 
the trend of the percentages indicates that as the time required from 
project initiation to a development go-ahead decision became longer, so 
did the support time requirement to reach a profitability situation. 
However, the added effect which can be inferred by this comparison of 
shifts of percentages was that any increase in development time caused a 
substantially greater increase in time to reach profitability. 

Table 4 shows this relation between these two factors. 

TABLE 4 

LEAD TIMES FROM INITIATION OF R&D EFFORTS 



Years 

Under 2 
2 to 4 
5 to 10 
11 to 20 



Percentage of Respondents Indicating Lead Time 

To Development To 

Go-Ahead Decision Profitability 



21.8 

44.4 

31.0 

2.8 



11.7 
32.4 

49.7 
6.2 
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Comparison of answers to these questions also inferred that if 
Question 8 had allowed a variance of time, or particularly a longer time, 
the percentage of successful efforts reported would probably have been 
greater. Since several responding companies indicated, as tabulated in 
Appendix A, that they did not conduct basic research, it was not sur- 
prising that many more "missing" data points were coded for Question 8 
than for Questions 9 and 10. 

Considering the three questions as a group, data were inter- 
preted as showing that a substantial proportion of industry willingly 
supported R&D efforts over sustained time periods in the expectation of 
eventual, but not guaranteed, payoffs. 



6 . Percentage of R&D Internally Funded 



Question 11. What percentage of your basic research effort is 
internally funded? 

(1) Less than 20% (4) 60 to 79% 

(2) 20 to 39% (5) 80 to 100% 

(3) 40 to 59% 



Question 12. What percentage of your applied research effort 
is internally funded? 

(1) Less than 20% (4) 60 to 79% 

(2) 20 to 39% (5) 80 to 100% 

(3) 40 to 59% 



Question 13. What percentage of your development effort is 
internally funded? 

(1) Less than 20% (4) 60 to 79% 

(2) 20 to 39% (5) 80 to 100% 

(3) 40 to 59% 

The literature survey in general indicated that a significant 



portion of industrial R&D was funded by other than the performing 



companies . 

Questions 11, 12, and 13 were designed to determine the level of 
internal funding, and also to discover any strong tendency to seek 
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outside funded effort. It was evident from the data that most of the 
industrial R&D effort was funded directly by the performing companies. 
Eighty-three percent of respondents conducting basic research internally 
funded more than 80 percent, while only 12 percent funded less than 20 
percent. A larger percentage, 84 percent, funded more than 80 percent 
of their applied research, and 88 percent funded over 80 percent of 
their development effort. A positive correlation was found between the 
company size and the companies that internally fund their basic research 
effort, as indicated by a Kendall Tau of 0.23 with a significance of 
0.001. This was interpreted to indicate that the larger the company, 
the more likely it is to conduct its own research. 

The survey responses to these questions are summarized in Table 
5. 



TABLE 5 

PERCENTAGE OF R&D EFFORT INTERNALLY FUNDED 



Percentage of 
R&D Funded 
by Respondent 


Percentage 


of Responding Companies 


Basic Research 


Applied Research 


Development 


Less than 20% 


11.8 


5.6 


4.1 


20 to 39% 


1.6 


0.7 


2.1 


40 to 59% 


2.4 


5 . 6 


2.1 


60 to 79% 


1.6 


4.2 


4.1 


80 to 100% 


82.7 


84.0 


87.6 
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D. SECTION THREE - GENERAL MANAGEMENT 



This section of the test instrument was designed to provide informa- 
tion on the management of R&D in the industrial sector for possible clues 
to a system that would improve Navy management effectiveness and efficiency. 

The federal government has been criticized many times for the manage- 
ment of resources under government control. One of the criticisms was: 
"federal-funded civilian research and development is not sufficient to 
bring about technological change in the private sector to any signifi- 
cant extent [Government Executive, June 1976^." This article reviewed a 
report by Arthur D. Little, Inc. which indicated that the federal govern- 
ment was doing very little to "push" and "distribute" technological in- 
novations to the industrial community where the major benefits could be 
realized. According to this report, for technological innovations to be 
implemented, convergence of six elements was required. The elements pro- 
posed were (1) knowledge generated through R&D, (2) user need, (3) an ad- 
vocate or champion, (4) availability of resources, (5) favorable risk 
factors, and (6) favorable timing. It was suggested that the federal 
government could supply some of the missing elements, such as resources 
needed and favorable risk factors to expedite the innovations. 

1. Primary Task Assignment 

Question 1. Selection and assignment of a specific R&D task 
to an organizational group or individual is based 
primarily on: 

(1) Availability of demonstrated project management 
expertise 

(2) Availability of technological expertise in-house 

(3) Formation of a new group incorporating both man- 
agement and technological expertise (includes 
organizational reassignment) 

(4) Availability of support personnel 
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(5) If capability not existant in-house, contract 
to best qualified management sub-contractor 

(6) If capability not existant in-house, contract 
to best qualified technologically qualified 
sub-contractor 

(7) Available in-house personnel with best back- 
ground adaptable with minimum training 

(8) Other 

Selection and assignment of a specific R&D task to a group or 
individual was based primarily on the availability of in-house technolo- 
gical expertise, according to 47 percent of the respondents. Twenty- 
five percent of the companies assigned R&D tasks by the availability of 
personnel with good background and adaptability to training for the spe- 
cific task. It was considered that background and adaptability for 
training reasonably represent an available technological capability 
needing only a relatively minor upgrading. Accordingly, it was believed 
that these two factors were sufficiently related to combine them for 
analysis. Seventy-two percent of the companies (the combination) 
assigned tasks on the basis of available in-house personnel either pos- 
sessing specific required technological background, or adaptability to 
be trained to perform the specific tasks. 

The fifth hypothesis contends that availability/evaluation of 
technological expertise is the major factor in selection of the activity 
or individual to pursue the research and development effort in more than 
70 percent of specific efforts. Accordingly, the above data were consi- 
dered as verifying acceptance of this hypothesis. The sixth answer 
offered for selection asked about choice of a sub-contractor on the 
basis of technological qualification. An additional six percent of re- 
spondents indicated this choice; therefore, the fifth hypothesis was 
considered as supported. 
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2 . Project Priority Criterion 



Question 2. Which R & D project objective would get highest 

priority at your company? 

(1) Reducing operating costs of processes 

(2) Timeliness (with respect to ) 

(3) Improving company ! s competitive position by 
providing better products 

(4) Probability of success of R & D project 

(5) Soundness of technological approach 

(6) Estimated cost 

(7) Product diversification for company growth 

(8) Other (specify) 

This question was to obtain an evaluation of R&D project selec- 
tion criteria among industrial companies and to compare the findings 
with the literature survey results. 

It is considered very important by most management personnel to 
define goals and objectives for the R&D effort within a company. Approx- 
imately 94 percent of companies performing a significant amount of R&D 
have specific objectives, however, only 79 percent have written objec- 
tives ^Seiler, 1965]. Most of these R&D objectives are formulated to 
satisfy six basic goals: 

"1. To ensure that the corporation will continue to operate in 
areas of growing business activity. 

2. To make the best possible use of the corporation’s resources, 
including scarce raw materials, technical specialties, and 
management talent . 

3. To utilize available markets and develop new ones to the 
fullest extent possible. 

4. To diversify the corporation’s activities for maximum 
stability. 

5. To ensure continuing and steadily increasing profits for the 
corporation . 

6. To contribute to the corporation’s ability to accept appro- 
priate responsibilities in the social environment in which 
it operates.” 
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It is believed that more effort should be given to defining more 
specifically R&D objectives in most companies. 

Industrial research projects have the objectives of (1) reducing 
operating costs, (2) improving the company’s competitive position by pro- 
viding quality products and technical assistance, and (3) developing new 
investment opportunities for the company and opening additional markets 
for new or present products jjloman, 1968^. Project selection depends on 
quantitative and qualitative factors that are very difficult to deter- 
mine in most cases. Mr. Roman very accurately described the project 
selection situation in his statement: 

"Everything involved in project selection is relative. It is 
nigh on impossible to establish a quantitative formula which takes 
all the subjective factors into consideration. From the many as- 
pects of a project and their varying effects, it is apparent that 
no set format will answer all management’s decision-making needs. 
Methodology can be employed as a screening process prior to the 
final decision. A quantitative approach will narrow the possibil- 
ities and provide a smaller field of focus, but the capability and 
intuition of the decision-maker may well be the most critical ele- 
ments in project selection.” 

R&D project selection in the government differs from that in pri- 
vate industry primarily because of the different objectives involved. 

In industry, the need and potential profit are the two most compelling 
factors affecting project selection, while in government, the driving 
force is mission accomplishment. These differences in objectives change 
the relative weights on other factors of project selection, such as pro- 
ject cost, cost savings and personnel requirements. 

’’The selection of projects upon which to expend resources in any 
company involves consideration of many factors. Some of these factors 
are requirement, soundness of technological approach, probability of 
project success, timeliness of the project, and estimated cost ^Das , 
1969^.” Das performed a study to see if reliable project selection 
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could be made based on these factors alone, and came to the conclusion 
that the measures of importance of projects probably have significant 
association with additional factors not considered in his study. 

There are certain criteria that can be used during the R&D plann- 
ing stage to measure the probable success of a research program. These 
criteria are normally associated with consideration of (1) goals of the 
sponsor, monitor, and researchers, (2) competence of the research per- 
sonnel, (3) value of the program and progress made, (4) the recognition 
granted by all parties concerned, and (5) specific results Qsalkovitz, 
Armstrong, and Howe, 1970^. Attempts to apply linear programming tech- 
niques to the problem of project selection for the allocation of re- 
search and development resources appear to have some merit JjTreeman and 
Gear, 1971^J. 

The data of this survey well supports the general objectives of 
industrial R&D as proposed both by Seiler and by Roman since improving 
the company's competitive position by providing better products was se- 
lected as the primary objective for the R&D effort by 54 percent of the 
respondents. Seventeen percent of the companies indicated cost reduc- 
tion as the primary objective, and eight percent indicated company growth. 

In addition to the respondents placing importance on cost reduc- 
tion, other respondents also agreed with the proposals made by Das 
[l969] ; seven percent indicated timeliness, and three percent cited 
probability of success. 

Numerous comments were offered by the eleven percent of the re- 
spondents who selected "other 11 as an answer. These comments, and addi- 
tional ideas by companies that selected specific offered answers to the 
question, were tabulated in Appendix A. Several of these comments 
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indicated that more than one or all of the suggested criteria were used 
in establishing project priority. As to what might be considered an 
opposite extreme, some comments indicated government regulations as a 
determining factor . 



3. Budget Commitment Stage 



Question 3. At what stage in the budget formulation process is 

the funding level committed for R & D? 

(1) Beginning of the planning phase 

(2) When sales estimates are made 

(3) When profit estimates are made 

(4) At end of budget allocation, providing funds 
are available 

(5) Other (specify) 

The importance of long range planning of R&D cannot be over em- 
phasized. Many authors feel that R&D planning should be for a minimum 
of five years and the planning process should follow seven major steps: 

"1. Specify as clearly as possible the basic technological objec- 
tives which are of primary interest. These may be stated 
broadly at the beginning but must eventually be broken down 
into particular areas. 

2. Identify the goals toward which the company is working, which 
ideally are set forth in the corporate and the research 
objectives . 

3. In keeping with the preceding two factors, seriously and imag- 
inatively consider all possible results which may be achieved 
if the research efforts are successful. 

4. Rank the hypothetical capabilities of the research efforts as 
determined in the preceding step, in terms of their potential 
contribution to the achievement of specific goals. The rank- 
ing should provide insight into the efforts needed to achieve 
the corporate or the research objectives. 

5. Outline the principal technological steps which are required 
to achieve the hypothetical research results listed in the 
preceding step. Such an outline should uncover the more sig- 
nificant gaps in existing knowledge. 
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6. Select the small number of high-value research results, as 
determined previously, which have reasonably well-specified 
steps and a minimum number of knowledge gaps. Give these re- 
sults further examination and consideration for full-scale 
research efforts. 

7. Remain alert to any developments that could significantly 
change the value ratings. used in the preceding steps. When 
research breakthroughs fill in missing knowledge gaps, the 
process should be repeated. " 

The accomplishments of the above steps may prove to be difficult 
in many cases, however, they are considered necessary for meaningful, 
realistic planning [^Seiler, 1965^j . 

Some of the major difficulties encountered in planning basic re- 
search are: (1) feedback is long delayed, (2) the research is not re- 

peated, (3) sometimes the attitudes of those working together on a 
project are at cross-purposes, (4) attitudes of those involved in re- 
search can hinder plans, and (5) research plans can be rendered obsolete 
by inventions elsewhere ^McGlauchin, 1968]. 

The planning and budgeting of R&D has been recognized almost uni- 
versally as a very difficult task and broad enough in scope to justify a 
study by itself. For purposes of this survey, the question was inten- 
tionally simplified and attempted only to determine the basic time in 
the budgeting process when commitment of funds is actually planned or 
specified to support R&D effort. 

Nearly one-half of the responding companies committed funds to 
R&D effort at the beginning of their budget planning phase. Research 
and development budgeting, therefore, was a prominent part of the fiscal 
planning cycle for this group. 

Sixteen percent (23 companies) indicated that R&D funding was 
committed when profit estimates were made. No attempt was made in this 
questionnaire to learn when profit or sales estimates were made relative 
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to other parts of the planning cycle. 

Ten percent (15 companies) indicated that R&D funding commitment 
was made when sales estimates were made. Of these 15 companies, only 
seven were among the group of 30 companies that had indicated planning 
as a percentage of sales in response to the earlier question. 

It appeared that the use of a controlling financial index for 
R&D budgetary planning factor was not necessarily coordinated with the 
development of the budgetary plan. This attested both to the complexity 
of the budgeting process, and the decisions related to R&D funding 
support . 

An additional 11 percent of the respondents committed R&D fund- 
ing at the end of the budget allocation process provided funds were a- 
vailable, and 13 percent determined R&D funding at other times. 

4. Equipment Acquisition Method 

Question 4. If you anticipate equipment requirements for R&D 

effort in addition to current R&D capability, 

your most common acquisition method is: 

(1) Purchase to establish or maintain general 
research capability 

(2) Purchase for specific R&D task 

(3) Transfer to R & D upon completion of other 
usage 

(4) Develop in-house as need arises 

(5) Other (specify) 

Some previous surveys or treatises on research and development 
have recognized the importance of making ,T the best possible use of the 
corporations resources ^Seiler, 196fT|. 11 No literature was discovered 
which presented results of inquiries into policy or procedure as prac- 
ticed to support R&D equipment requirements. 
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"Successful research and development revolve around the crea- 
tive, talented scientist. Nothing useful or valuable can be done 
without the trained, talented, innovative human brain. 

But the researcher can be severely limited, and at times pre- 
vented from making progress, if his research tools are inadequate 
(^Industrial Research, January 19 75^." 

Less than two percent of responding companies in this survey 
transferred equipment within house for R&D uses. This indicated that 
nearly all companies obtain and commit needed equipment directly for R&D 
support . 



Fifty- two percent of the companies purchased equipment to per- 
form specific R&D tasks. Thirty-five percent purchased equipment to es- 
tablish or maintain general research capability, seven percent developed 
their own equipment for R&D, and about four percent obtained R&D support 
equipment from other sources. 



5 . Funds Reprogramming Criterion 



Question 5. What is the most important criteria of your policy 
regarding reprogramming of R & D funds? 

(1) As new opportunities or ideas are generated 

(2) Upon failure of existing R&D projects to 
materialize useful results/products 

(3) Based on periodic review of all ongoing efforts 

(4) Observed/predicted major shift of competitor's 
direction of effort 

(5) Other (specify) 

The need for and capability to fund a research and development 

effort which occurs between established or normal budget cycles has led 
to many difficult decisions such as which projects to cut. 

In one related discussion of R&D budgeting, the importance of 
asking eight basic questions of a candidate project for research and de- 
velopment fund allocation were specified: 

"l. Can it be done? 
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2. Can the company do it? 

3. Are the necessary resources likely to be available to success- 
fully manufacture and market the product or to implement process 
improvements ? 

4. Is there a real demand or can one be created? 

5. Can the product be profitably made and sold at a price which 
assures steady demand? 

6. Is the company capable of successfully marketing the product? 

7. What is the competition doing and how will it react? 

8. How do the potential benefits compare with the apparent risks 
|~Jones, 1973]?'' 

Question 5 attempted to determine which of the suggested, or 
other, reasons or policies for M between the cycle" funding changes was 
most used. 

The reprogramming of R&D funds was accomplished mainly at period- 
ic reviews by 61 percent of the companies. New opportunities or ideas 
were the basis for an additional 31 percent of the companies initiating 
funds reprogramming action. The remaining eight percent cited other rea- 
sons for reprogramming , including project failure and the competitive 
efforts of other companies. Each had less than three percent. 



6 . Personnel Assignment Method 



Question 6. How are most personnel assigned to R & D projects? 

(1) Hired and trained especially for the specific 
R&D effort 

(2) Assigned as a result of past performance in 
other area 

(3) Assigned from permanent R&D staff 

(4) Hired to acquire specific technological 
capability 

(5) Other (specify) 

This question was intended to further clarify the acceptance or 

rejection of the fifth hypothesis regarding the importance of technologi- 
cal expertise, and to find if there were any predominant personnel poli- 
cies in industrial practice. 

The data indicated that personnel were assigned to R&D projects 
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from their permanent R&D staff by 59 percent of the companies, while an 
additional 22 percent indicated the R&D personnel assignments were made 
on the basis of past performance. Upon review, it was apparent that 
these two categories could be easily confused or both applicable in many 
cases. Therefore, no additional conclusions were made regarding the hy- 
pothesis. Over eight percent of the companies hired personnel for spe- 
cific R&D tasks, six percent hired personnel for specific technological 
expertise, and four percent had other reasons for assigning personnel to 
R&D projects. 

7 . Percentage of R&D Contracted-Out 

Question 7. What percentage of your R&D effort is sub- 
contracted out? 

Less 

than 5% 5-25% 26-50% 51-75% 76-100% 



Basic research 


(i) 


(2) 


(3) 


(4) 


(5) 


Applied research 


(i) 


(2) 


(3) 


(4) 


(5) 


Development 


a) 


(2) 


(3) 


(4) 


(5) 



During the six years preceding 1973, Industrial R&D Laboratories 
in England had grown at a 25 percent per year rate. These laboratories 
perform research and development for different companies which increase 
the emphasis on ethical conduct of the laboratories [[ Sharpies , 1973^* 

The literature suggested that, in England, an increasing number of com- 
panies were contracting-out some or most of their R&D effort. The above 
question was intended to obtain information on the amount and type of 
R&D effort contracted-out in the United States. 

More than three-quarters of the responding companies contracted- 
out less than five percent of their total R&D effort in any category. 

The percentage of companies responding to this question in each R&D cat- 
egory with the indicated percentage of effort contracted-out is shown in 
Table 6. 



47 



TABLE 6 



PERCENTAGE OF R&D EFFORT CONTRACTED-OUT 



Percentage of Percentage of Responding Companies 



R&D Effort 
Contracted-out 


Basic Research 


Applied Research 


Development 


Less than 5% 


76.9 


77.6 


78.5 


5-25% 


10.4 


14.0 


13.9 


26-50% 


3.7 


4.9 


3.5 


51-75% 


2.2 


0 


0.7 


76-100% 


6.7 


3.5 


3.5 



Table 6 shows that less than one-fourth of the companies con- 
tracted-out any significant proportion of their basic research. A rela- 
tively strong negative correlation was found between company size and 
basic research contracted-out, as indicated by a Kendall Tau of -0.25 
with a significance of 0.001. This shows the tendency of the larger com- 
panies to perform their basic research within their own facilities. 

A smaller percentage of the overall applied research and develop- 
ment was contracted-out than the basic research, as nearly 80 percent of 
the companies contracted-out less than five percent of their applied 
effort. The larger companies also performed in-house a larger percentage 
of both categories, applied research and development, than the smaller 
companies as verified by Kendall Taus of -0.20 and -0.22 respectively, 
each with a significance of 0.001. 

One-half of the companies contracting-out more than 50 percent of 
their R&D effort were utility companies. One of these companies offered 
as a general comment "The R&D work among the electric utilities, 
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especially that conducted by the Electric Power Research Institute, 
constitutes a special case ..." 



8 . Percentages of Externally Funded R&D 



Question 8. What percentage of total R&D effort performed by 
your company is externally funded? 

Less 



than 5% 

Government 


5-25% 


26-50% 


51-75% 


76-100% 


basic research 
Government 


a) 


(2) 


(3) 


(4) 


(5) 


applied research 
Government 


(i) 


(2) 


(3) 


(4) 


(5) 


development 
Other commercial 


a) 


(2) 


(3) 


(4) 


(5) 


basic research 
Other commercial 


a) 


(2) 


(3) 


(4) 


(5) 


applied research 
Other commercial 


a) 


(2) 


(3) 


(4) 


(5) 


development 


(i) 


(2) 


(3) 


(4) 


(5) 



Tables 7 and 8 show the percentage of responding companies con- 
ducting research and development effort in the various categories funded 
by government or other commercial sources respectively to the extent 
indicated . 



TABLE 7 

EXTERNAL R&D FUNDING BY GOVERNMENT SOURCES 



Percentage of 
Externally Funded 
R&D 


Percentage of Responding Companies 


Basic Research 


Applied Research 


Development 


Less than 5% 


91.0 


82.8 


81.6 


5-25% 


1.5 


6.7 


7.4 


26-50% 


2.3 


3.7 


2.9 


51-75% 


0.8 


3.0 


2.9 


76-100% 


4.5 


3.7 


5.1 
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TABLE 8 



EXTERNAL R&D FUNDING BY COMMERCIAL SOURCES 



Percentage of Percentage of Responding Companies 

Externally Funded 



R&D 


Basic Research 


Applied Research 


Development 


Less than 5% 


93.1 




89.6 


87.6 


5-25% 


1.5 




3.7 


5.1 


26-50% 


2.3 




2.2 


2.2 


51-75% 


0 




0 


2.2 


76-100% 


3.1 




4.4 


2.9 


Approximately 


two-thirds 


of the 


respondents to this 


question 


indicated less than five percent 


external funding in all six 


categories. 



Two companies reported 76 to 100 percent in all six categories. 



9 . Trends of Externally Funded R&D 



Question 9. Of those efforts indicated in question 8 above, the 
following past five year trends have occurred: 



Increased Decreased Varied Stable 



Government 



basic research 
Government 


(i) 


(2) 


(3) 


(4) 


applied research 
Government 


(i) 


(2) 


(3) 


(4) 


development 
Other commercial 


(i) 


(2) 


(3) 


(4) 


basic research 
Other commercial 


(i) 


(2) 


(3) 


(4) 


applied research 
Other commercial 


(i) 


(2) 


(3) 


(4) 


development 


(i) 


(2) 


(3) 


(4) 



The histograms representing responses to Question 9 are 



summarized in Tables 9 and 10. 
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TABLE 9 



PERCENTAGE TRENDS IN EXTERNALLY FUNDED R&D 



R&D Category 




^Percentage of Responding Companies 




Increased 


Decreased 


Varied 


Stable 


Government basic research 




9.7 


16.8 


3.5 


69.9 


Government applied research 




14.8 


13.0 


3.5 


68.7 


Government development 




21.4 


9.4 


6.0 


63.2 


Other commercial basic research 


11.5 


11.5 


4.4 


72.6 


Other commercial applied research 


18.6 


5.9 


5.9 


69.5 


Other commercial development 




22.1 


9.8 


5.7 


62.3 


*Missing data omitted. 














TABLE 10 






/ 


NUMERICAL TRENDS 


IN EXTERNALLY FUNDED R&D 






R&D Category 




Number of 


Responding 


Companies 




Increased Decreased Varied 


Stable Missing* 


Government basic research 


ii 


19 


4 


79 


34 


Government applied research 


17 


15 


4 


79 


32 


Government development 


25 


11 


7 


74 


30 


Other commercial basic 
research 


13 


13 


5 


82 


34 


Other commercial applied 
research 


22 


7 


7 


82 


29 


Other commercial development 


27 


12 


7 


76 


25 



^Respondents returning totally blank questionnaires not included. 
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The computer input data listing showed missing data in all cate- 
gories for 21 respondents. For nearly all of these 21 companies, com- 
ments on the respective questionnaires indicated either that they do not 
conduct basic research, do not conduct any research, or internally fund 
80 to 100 percent of all R&D. Therefore, trends in external funding 
were not appropriate for them. These comments were typical of several 
on other respondents’ questionnaires wherein partial missing data were 
noted. The nature of at least one of the "exception" companies led to 
the conclusion that their research and development efforts were related 
to product markets and advertising. 

Table 10 was included because inspection of the data base showed 
such a large amount of missing data for this question that the trend 
shown in Table 9 might be considered misleading or erroneous. The gen- 
eral trend shown in Table 10 agrees with the previous table thereby ver- 
ifying that the missing data does not affect the results or analysis. 

Both tables showed that the majority of companies reported stable 
research and development efforts. In fact, the input data also listed 
58 companies that indicated stable externally funded R&D levels in all 
six tabulated categories. This represents approximately 40 percent of 
the total responding companies. 

For further analysis, those companies that responded "increased" 
or "decreased" were considered by R&D category as a group. Tables 9 and 
10 show a net decrease in government funded basic research, a balancing 
net effect in externally funded commercial basic research, and net in- 
creases in the other four categories. These data were considered to 
further support the first hypothesis which predicted a net increase 
trend of industrial companies, and was in agreement with the results 
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noted in Question 1 of Section Two which also showed a net increasing 
trend in total R&D by respondents. 

10 . R&D Evaluation Criterion 

Question 10. What technique do you consider the most important 

in evaluating the return on investment or value 

of your overall R&D program? 

(1) Number of commercial products developed 

(2) Number of published technical articles 

(3) Number of patents issued 

(4) Cost savings estimates 

(5) New process developments 

(6) Market share trend 

(7) Other (specify) 

Various methods of evaluating R&D performance have been proposed, 

such as evaluating the individual performing the research, evaluating 
the group involved in the research, evaluating the organization perform- 
ing the research, evaluating the product developed, and evaluating on a 
national basis. In evaluating the individual, techniques such as com- 
parison with other organization members, ranking within the organization, 
weighted performance characteristics and the number of publications per 
individual are used. One method of group evaluation is to sum up the 
assessments of the individuals within the group; however, this can be 
misleading. Growth and profitability are usually the two standards for 
measuring the organization's performance; however, other factors con- 
tribute significantly, such as effectiveness and efficiency. Evaluating 
a developed product can be done from various viewpoints; however, it is 
usually evaluated for the contribution to the organization's objectives. 
Evaluating R&D on a national basis is usually very difficult; however, 
performance can be approximated if accomplishments can be identified and 
compared with original objectives and estimated costs Q Roman, 1968^. 



53 



Dr. William Thurow, of MIT, states that both civilian and mili- 
tary expenditures should be judged only on the direct benefits that they 
are supposed to be producing, and the spinoffs and economic multipliers 
should always be ignored in evaluating projects and allocating funds 
^Long, 1976^. This stand is very different from that taken by another 
senior economist who states that every one billion dollars spent in NASA 
R&D programs return approximately fourteen billion dollars over a ten 
year period, including spinoffs and related benefits ^Gilison, 1976^]. 

The above two cases indicated the difficulty in evaluating the 
value of R&D effort. Many industrial companies feel R&D is necessary 
for survival and prefer to evaluate the R&D programs subjectively while 
others attempt a quantified evaluation. 

A suggested procedure for evaluating R&D is to attempt to: 

"1. Determine the amount of profit produced and capable of being pro- 
duced by each research and development project. 

2. In the case of profits not yet materialized, discount the profits 
of future periods because of (a) the factor of uncertainty, (b) 
the prospect of changes that might reduce the profitability, and 
(c) the cost of money and other factors. 

3. Apply a money value to other services performed by the research 
and development group, based on a market value agreed upon between 
the research and development staff and the department receiving 
the service. 

4. Total the expenditures of the research and development group. 

5. Subtract the total expenditures from the sum of discounted profits 
produced and services produced. If the result is a plus figure, 

it tends to indicate that the activity has been effective T Johnson, 
1972]." 

The difficulty of implementing the above procedure has been in 
arriving at quantified values for the steps of the procedure. Many times 
data were not available, probabilities of sales forecasts were wild 
guesses, and other values were products of wishful thinking. A study on 
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evaluating research and development by Professor J. B. Quinn led to the 
conclusion that R&D should be evaluated by a "segmental approach" [[Quinn, 
September 1959^]. This evaluation procedure called for a technical eval- 
uation, an economic evaluation, and a management evaluation periodically 
to review each R&D project. Specific information was required for pro- 
ject reviews including (1) each program f s cost to date, (2) what it had 
accomplished technically, (3) the value of this technology, (4) what in- 
vestigations were projected, (5) the potential value of each and (6) how 
likely each was to succeed, economically and technically. Even when the 
segmental approach was utilized, no generally useful over-all formula 
for "how much to spend on R&D" was developed. Such guiding ratios as 
expenses to sales, expenses to capital, this year’s to last year’s ex- 
penses, and measured over-all return on R&D investments were of little 
value in research planning, according to Professor Quinn. 

"Number of published technical articles" and "patents issued" 
were included as evaluators in some discussions of research and 
development . 

The "other” answer in this question was included to attempt to 
discover additional evaluation factors which industry considered important. 

A small number of respondents selected a " most important" factor 
of R&D program evaluation, and then added comments indicating additional 
equally or nearly as important factors. More than one-fourth of the ad- 
ditional respondents selected "other", and then offered specific comments. 
Appendix A includes a tabulation of all voluntary comments. These vol- 
untary comments were analyzed and the two most numerous types of re- 
sponses, "profit" and "other financially quantifiable," were computed. 

The total resulting responses received were organized as presented in 
Table 11. 
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TABLE 11 



MOST IMPORTANT FACTORS IN R&D EVALUATION* 



Factor 


Number of 
Responses 


Percentage of 
Total Responses** 


Number of products 


48 


33.1 


Published articles 


0 


0.0 


Number of patents 


0 


0.0 


Cost savings 


18 


12.4 


New process developments 


17 


11.7 


Market share trend 


25 


17.2 


Profit 


18 


12.4 


Other financially quantifiable 


7 


4.8 


Other 


12 


8.3 


Missing 


2 





^Voluntary comments included. 
**Missing values not included. 



Approximately one-third of the respondents indicated the "num- 
ber of commercial products developed" as being of prime importance. If 
the companies that selected "market share trend" (the second most fre- 
quently selected) and "number of commercial products developed" were 
combined as an indication of interest in maintaining a competitive posi- 
tion, this combined group represents one-half of the responding companies. 

The next most frequently indicated evaluation factors were "cost 
savings," from the original question list, and "profit," discovered among 
the "other" comments. Each of these evaluation techniques or factors was 
indicated by 12 percent of the companies. 

It was noted that only one company indicated in a comment that 
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"published technical articles" and "patents issued" were important as 
evaluative factors, but in that specific case neither was first in im- 
portance for that company. 

When only cost savings, profit, and other financially quantifi- 
able factors were considered, a total of 29 percent of the companies 
were represented. Number of commercial products developed, new process 
developments and market share trend were considered numerically quanti- 
fiable as to effect on company goals achievement. With existing account 
ing techniques, these last three factors were believed to be financially 
assessable as well. The factors discussed above as financially quanti- 
fiable were combined and the total effect was calculated. On this basis 
over 90 percent of the responding companies indicated that some finan- 
cially quantifiable benefit was the most important method of evaluating 
R&D effort. 

The sixth hypothesis contended that financially quantifiable 
benefit of R&D effort is the most important factor in 60 percent of in- 
dustrial evaluations. Therefore, the sixth hypothesis is accepted. 

Two "other" comments were noted particularly because of the 
wide divergence between them. One respondent offered the evaluation 
formula of "sales of new products + 5x cost savings before tax + 5x 
royalties"; another, while indicating some factors are used, seemed to 
infer frustration or uncertainty by the comment "no real good technique 
ever found." 
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IV. VALIDATION OF HYPOTHESES 



Six hypotheses were proposed as characteristics which, based on the 
literature reviewed, were expected to be confirmed by the data resulting 
from the survey of industry. 

Contrary to expectations, the survey responses clearly rejected the 
second hypothesis, and marginally rejected the fourth hypothesis. The 
appropriate null hypotheses were accepted as stated in the analysis of 
results . 

Within the rationale presented in the analysis of results, the other 
four hypotheses were accepted as validated by the survey results. 

For clarity, the accepted hypotheses are presented as follows: 

1. The trend of R&D effort levels in general is increasing at a net 
of ten percent of industrial corporations. 

3. For two-thirds of sponsoring companies, less than five percent 
of basic research efforts yield financially profitable results 
within five years. 

5. Availability /evaluation of technological expertise is the major 
factor in selection of the activity or individual to pursue the 
research and development effort in more than 70 percent of 
specific efforts. 

6. Financially quantifiable benefit of R&D effort is the most im- 
portant factor in 60 percent of industrial evaluations . 

The rejected hypotheses are: 

2. More than 50 percent of industry R&D efforts funding is based on 

a percentage of some financial index of the corporation concerned. 

4. For two-thirds of performing companies, the majority of research 
efforts yielding results warranting follow-on development of new 
products, did so within four years of initiation. 
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V. PRINCIPAL FINDINGS AND CONCLUSIONS 



The following principal findings and conclusions were derived from 
the survey response data as discussed in the analysis of results: 

1. The percentage of responsive answers to the questionnaires dis- 
tributed was greater than that of most analogous surveys reported in the 
literature reviewed. All segments of the survey population as measured 
by the number of laboratories operated, the company size and the indus- 
try classification were represented by data inputs. Therefore, it was 
concluded that the data analyzed was representative of the complete in- 
dustrial frame specified for the survey. 

2. The high percentage of executive interest in responding to the 
survey, as indicated by the titles of the responding officers, was con- 
cluded to confirm substantial validity of the policy indications shown 
by the data returned. 

3. The respondents’ perceived trends of R&D effort over the past 
five years strongly supported the first hypothesis which predicted an 
increasing trend. 

4. Sixty-two percent of responding companies were classified by a 
National Science Foundation definition as not R&D intensive; that was, 
less than two percent of their total operating budget was devoted to 
research and development. 

5. Sixty-one percent of the responding companies did not use any 
type of financial data index in the process of budgeting R&D funding 
levels. Accordingly, the null of the second hypothesis was accepted. 

6. Of those companies that did use a financial index as an R&D budg- 
et basis, nearly two-thirds used a percent of sales as a controlling 
factor. 
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7. Eighty percent of reporting companies allocated less than five 
percent of their R&D effort to basic research. Nearly the opposite ex- 
treme, 60 percent of the companies devoted at least 60 percent of their 
R&D effort to development. The two factors rated as most important in 
evaluation of R&D effort were the number of new products developed and 
market share trend. When considered together with the above effort dis- 
tribution, it was concluded that most companies devote the majority of 
their R&D effort toward maintaining or improving their position within 
their competitive industry as they perceive it. 

8. Seventy-one percent of responding companies indicated that less 
than five percent of their basic research yielded financially profitable 
results within five years, thereby confirming acceptance of the third 
hypothesis. The responses of one-third of the companies confirmed accep- 
tance of the null of the fourth hypothesis by indicating that the majori- 
ty of research efforts pursued which yield results warranting follow-on 
development of new products required more than four years from initiation. 
Lead time to profitability was indicated as five to ten years from ini- 
tial research by nearly 50 percent of respondents. From these facts, it 
was concluded that a substantial proportion of industry supported R&D 
efforts over sustained time periods. 

9. More than 80 percent of the companies reporting in each of the 
three defined categories of research and development indicated that more 
than 80 percent of their efforts were internally funded. 

10. More than 70 percent of the companies assigned tasks based on 
available in-house personnel either possessing the required technological 
background or who were adaptable with a minimum of training. This data 
was considered to be conclusive support for accepting the fifth hypothesis. 
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11. Fifty-four percent of the reporting companies indicated improv- 
ing the company ! s competitive position by providing better products was 
the primary objective of their R&D effort. 

12. Nearly one-half of the responding companies committed the funding 
level for R&D effort at the beginning of their budget planning phase. 

13. Fifty-two percent of the companies purchased equipment as needed 
for specific tasks. Thirty-five percent purchased equipment to establish 
or maintain a general research capability. It was concluded therefore 
that each method was significantly used. 

14. The periodic review cycle was indicated by 61 percent of the re- 
porting companies as their most important policy criteria regarding re- 
programming R&D funds. 

15. More than three-fourths of companies reporting in each category 
contracted-out less than five percent of their R&D effort. Less than 
five percent external funding by either government or commercial sources 
was reported in each category by more than 80 percent of industry. When 
all budgeting and funding trends were considered together, it was con- 
cluded that the majority of the industrial companies surveyed generally 
preferred to conduct their own R&D in-house, and to accept relatively 
little R&D effort from outside sources. 

16. Industry used a variety of factors as evaluators of return on in- 
vestment or value of their R&D program. Over 90 percent of the factors 
indicated were considered quantifiable. With existing accounting tech- 
niques, those factors were believed to be financially assessable. 
Accordingly, the sixth hypothesis was accepted. 
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VI . RECOMMENDATIONS 



The following actions are recommended: 

1. That the principal findings and conclusions presented herein be 
adapted and considered as possible guidelines for decision criteria for 
use where appropriate, potentially in Navy research and development 
policy matters . 

2. That a more in-depth study of the same industrial frame be con- 
ducted to discover related financial level data for coordination with 
the results presented herein, 

3. That a parallel study be conducted relative to Federal, academic 
institution and non-profit (or not-for-profit) private organizations. 

4. That the Results of this study of industry and any results of 
those studies recommended above be combined to formulate a comprehensive 
research and development decision model. 
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APPENDIX A 



Tabulation of Voluntary Comments 

Several questions of the survey instrument included an answer 
such as "other" which the respondent might select if no otherwise 
suggested answer properly fitted his company's situation. These, 
and some additional answers, clearly invited or asked for a vol- 
untary comment. The goal of this procedure was to clarify any 
indefinite answers. Also, if any particular answers should happen 
to be given a significant number of times, an important factor 
would be discovered which could have been overlooked otherwise. 

This procedure, for example, led to recognition of the relative 
importance of profit as an R & D effort evaluation method. Addi- 
tional comments were sincerely invited at the end of the 
questionnaire . 

All of the comments were considered during the analysis of 
responses. Recoding and re-analysis of answers to a particular 
question was done when the voluntary responses indicated a probable 
noticeable affect on the analysis of results. 

All of the comments are tabulated in this appendix so that any 
reader may interpret any inference or application from them to his 
own company's procedures or policies. 

To simplify and clarify presentation, and to aid in most mean- 
ingful interpretation, all voluntary answers or comments to a par- 
ticular question are tabulated as a group. The questions are 
grouped according to the sections of the survey. The number of the 
question, and the question to which the comments were offered, is 
given at the beginning of the listing of comments to which it 
applies. For example, "SECTION TWO, Question Three" means that the 
list of comments following that heading all were offered in response 
to the third question of Section Two. In any list, the column head- 
ed "Serial No." gives the serial number of the questionnaire from 
which the comment following the number was recorded. If a partic- 
ular comment is preceded by a number in parentheses, the number 
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in parentheses shows the answer to the particular question which 
the respondent selected before offering the comment. In all cases 
where no such number is shown, the respondent had either selected 
the last answer to the question (usually "other") or had added the 
comment in a general manner even though a comment had not been 
specifically sought. All comments offered by any particular 
respondent can be noted by correlating comments by questionnaire 
serial number from the several lists. All comments are quoted 
exactly from the questionnaire referenced by the serial number. 



SECTION TWO - BUDGETARY CONSIDERATIONS 
Question 1: 

How do you perceive your company's R Sc D effort over the past 
five years? 

(1) No significant change (3) Decreased significantly 

(2) Increased significantly (4) Suspended R & D efforts 

Serial No. Comment 

589 (2) (in dollars, but static as % of sales) 

Question 2: 

What percentage of your total operating budget does your R Sc D 
effort constitute? 

(1) Less than 2% (3) 5 

(2) 2 to 4% (4) 8 

Serial No. 

257 If "your" means my 

means my company's 

274 NA 

Question J: 

Annual R Sc D budgetary level is planned as: 

(1) A percentage of net sales 

(2) A percentage of net profit before taxes 

(3) A percentage of net profit after taxes 

(4) A percentage of total operating budget 

(5) Specified other corporate financial index 

(6) Limited to active contracts; i.e., no in-house development 

(7) Not related to a financial index or factor of corporate 

operations. General basis is , 



to 7% (5) Greater than 10% 

to 10% 

Comme nt 

own department, then 100%. If "your" 
corporate budget, then 5 to 7%. 



ial 

003 

004 

006 

021 

022 

024 

036 

051 

056 

059 

066 

069 

071 

076 

091 

097 

093 

112 

113 

129 

131 

136 

141 

142 

143 

153 

155 

157 

163 

166 

173 

175 

177 

178 

179 

182 

197 

203 

213 

216 

237 

249 

257 

274 

278 



Comment 



program need 

Need of product programs 

technical needs 

perceived need 

as justified 

.15 mills/kwh 

Project content Sc value 

Project basis 

need St financial health 

whim 

need 

opportunity 

dependent on project interest 

Approved research program plus necessary but unantici- 
pated research projects related to system operation 
need 

annual plan 
as needed 

Previous level + trends 
corporate management spells out 

judgement based on performance and business conditions 

Potential of each project 

Perceived need 

Size of Staff 

need for project efforts 

Justification of need on individual project basis 
number of good opportunities 

continuing need for product improvement and development 
of new business opportunities 
number and value of sugg. projects 
need 

need Sc value 
Total cost of R & D 

level of effort expressed as dollars identified in 
development plans prepared by operating divisions 
historical 

Corporate needs 8c opportunities 
1-4 above plus 

need for new technology in identified areas 

status quo 

Divisional Needs 

profit potential of the R 8c D 

need 

discretionary 

effort required to meet company ^oals 

judgement whether (1) number of researchers we have is 
adequate, too low, or too high; (2) by project eval- 
uations for potentials 
resource requirements 
Need For Technology 
% above previous yr. spend 
% pretax before R 8c D allocation 
what opportunities we identify 



65 



Question 3 (continued): 



Serial No. Comment 

30*3 $ needed to accomplish program objectives 

309 project and service activity 

310 NO. of good objectives 

315 5 year plans 

320 need 

323 as required by operating divisions 

325 assigned personnel and recognized need 

326 need 

328 RD & E as a % of net sales 

343 project or work forecast 

351 projected profit 

359 need for growth plans 

360 continuity and opportunity 

377 need 

380 anticipated requirements 

397 Division requirements 

418 approved projects 

423 As needs indicate 

469 Programs 

472 on evaluation of needs & opportunities 

478 not budgeted 

485 varied 

497 need 

501 long term % of profits 

503 need 

520 scope of programs at time of budget review - zero 

base budgeting 
524 project merit 

544 anticipated new product workload 

545 customer demand 

576 need 

586 Market technical requirements 

589 (1) (+ military products) 



Question 4: 

Total R & D budget is limited to: 

(1) Not to exceed 2 % of the above indicated financial index 

(2) Not to exceed 4% of the above indicated financial index 

(3) Not to exceed 7/» of the above indicated financial index 

(4) Not to exceed 10% of the above indicated financial index 

(5) Other 



Serial No. 
u04 

022 

028 

056 

066 

076 



Comment 

no specific limit. However, influenced by percent of 

net sales 

as justified 

not limited 

none 

Offer services to the Div. & Corp. as required 
as approved annually by management 
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Question 4 (continued): 



Comment 



Serial 

091 

098 

113 

131 

132 
136 
145 
153 

155 

166 

172 

173 

175 

177 

178 

179 
182 
197 
203 
216 
225 
257 

278 

289 

307 

308 
310 
315 
320 
323 
325 

328 

337 

343 

351 

359 

360 
377 
380 
418 
423 

472 

473 
497 
503 
520 
524 



No, 

need 

determined at beginning of each fiscal year, 
not related to a percentage 
not applicable 

Subject to divisional support 
See #3 above (perceived need) 
not applicable 

not limited by financial index but runs under 4"o of 

net sales 

not applicable 

established amount for each year 
varies 

negotiated ceilings 8c available earnings 

No present limit - but budget is established as stated 

above 

No financial index 
Management Judgement 
variable 

Not limited. Based on reasonable need. 

Financial justifications 
Divisional needs 
not limited 

Finance index and operating need 

Judgement on adequacy of research to meet perceived 
future needs 
no fixed amount 

30% of pretax before R & D allocation 
No stipulated limit 

determined by sum of program objectives 
Secondary consideration about 3*5 - 5 % of sales 
Ability to meet objectives 
not limited 

no specific limitation 

work to the capacity of assigned personnel and recog- 
nized need. 

RD & E = 2.5/* 

Confidential 
work approved 

constant man or person power 
No arbitrary guidelines 
adjustments for inflation 
amount of $ available 
effect on overhead rates 
N/A 

no index 

no specific limit 
not budgeted 
need 

not limited 
maintain budget 
no specified limit 
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Question 4 (continued): 



Comment 



Serial No. 

576 no budget limit 

577 Approval of active projects 

589 each division has own guidelines (varies from 2 % to 10%) 



Questions 5 through 13: 

Questions 5 through 13 asked for percentages of funding, times 
to success or similar factors related to separating results of 
basic research, applied research and development. The comments 
given by those companies that did comment were so similar, and 
generally applied to more than one question identically, that all 
the comments are presented as a group for simplicity. 



Serial No. 

029 

O 36 

056 

066 

091 

098 

225 

343 

497 

520 

589 



Comment 

Ambiguous: 0% internally funded, 0% externally funded 

(No provision for indicating, 0 basic research) 

N/A 

No basic research as defined 
No basic research done (0) 

No basic research 

Don't have any basic rsch 

Experience too limited to answer. 

not applicable 

None or N/A 

none 

too unusual for statistical reply 



SECTION THREE - GENERAL MANAGEMENT 

Question 1: 

Selection and assignment of a specific R & D task to an organi- 
zational group or individual is based primarily on: 

(1) Availability of demonstrated project management expertise 

(2) Availability of technological expertise in-house 

(3) Formation of a new group incorporating both management and 
technological expertise (includes organizational 
reassignment) 

(4) Availability of support personnel 

(5) If capability not existant in-house, contract to best 
qualified management sub-contractor 

( 6 ) If capability not existant in-house, contract to best 
qualified technologically qualified sub-contractor. 

(7) Available in-house personnel with best background adaptable 
with minimum training 

( 8 ) Other 

Serial No. Comment 

009- Depending on project, could be 1, 2, 3» 7. 

019 varies 7 , 2 , 6 in order 
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Question 1 (continued): 



Serial No. 
024 
084 
098 

177 

178 



Coimaent 

# 2 , 5 , 

Responsibility of organization or individual 
Automatically to engineering dept. 

Nature of assignment 

Capability, availability & overall priorities (8) 



Question 2 : 

Which R & D project objective would get highest priority at 
your company? 

(1) Reducing operating costs of processes 

(2) Timeliness (with respect to ) 

(3) Improving company's competitive position by providing 
better products 

(4) Probability of success of R & D project 

(5) Soundness of technological approach 

( 6 ) Estimated cost 

(7) Product diversification for company growth 

( 8 ) Other (specify) 



Serial No. Comae nt 



004 


Depends on all factors listed plus others 




019 


All above 




021 


No simple answer adequate 




044 


( 2 ) tech need 




066 


( 3 )t 7 a close second 




076 


( 2 ) company objectives 




089 


New method of producing natural gas 




100 


New products 




113 


New supplies of natural gas 




142 


( 2 ) government regulations 




153 


Generate highest profit potential. Includes 


1 , 2 & 3 . 


167 


size of market for product to be developed 




178 


Potential for successful new product 




213 


net profit potential 




225 


( 2 ), paper flow & 3 * 




278 


(1) Answers (3)» (^)* ( 6 ) or (7) may be of hi 
priority in many projects 


ghest 


305 


depends on profit centers needs & cost or risk or 




benefit ratios 




310 


balance of 1 , 3» 4, 5 




325 


( 2 ) Government Agency Permits and Regulatory 


deadl ines 


359 


potential value of project - profit 




360 


no single item controlling 




397 


varies with division 




423 


most pressing need 




465 


( 2 ) resource availability 




478 


( 2 ) competition 




544 


( 2 ) line extensions of existing products 
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Question 3: 

At what stage in the budget formulation process is the funding 
level committed for R & D? 

(1) Beginning of the planning phase 

(2) When sales estimates are made 

(3) When profit estimates are made 

( 4 ) At end of budget allocation, providing funds are available 

(5) Other (specify) 



Serial No« 
004 

024 

028 

076 

089 

172 

177 
■ 178 
213 
278 
325 

359 

360 
397 
478 
519 
524 
544 
586 



Question 4: 



Comment 

At end, but given same consideration as other functions. 
Reviewed annually or more often if changes take place. 
Public Service Commission Determination 
Total Integrated Planning Cycle 
after approval of R & D projects 
as required 

as part of a total corporate program on budget 

formation 

annually 

when proposed programs (total) are reviewed by mgt. 
when profit net potential defined 
Subject to Critical Review at all Phases. 

Not fiscally budgeted, see II - 3* 
same time as total co. budget approved. 

Separate budget submitted for Board Approval 
after approval of project justification 
not budgeted 
never 

at end of yearly planning cycle 
tied to fiscal year 

When total corp financial plan is prepared for Board 
of Directors 



If you anticipate equipment requirements for R & D effort in 
addition to current R 8c D capability, your most common 
acquisition method is: 

(1) Purchase to establish or maintain general research capa- 
bility 

(2) Purchase for specific R 8c D task 

(3) Transfer to R 8c D upon completion of other usage 

(4) Develop in-house as need arises 

(5) Other (specify) 



004 

112 

249 

360 

501 

542 



Comment 

Combination of 1 , 2 8c 4. 

1+4 above 

Lease with purchase option 
combination of 1 and 2 
varies considerably with circumstances 
Varies with t lie project 
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Question 5: 



What is the most important criteria of your policy regarding 
reprogramming of S & D funds? 

(1) As new opportunities or ideas are generated 

(2) Upon failure of existing R & D projects to materialize 
useful results/products 

(3) Based on periodic review of all ongoing efforts 

(4) Observed/predicted major shift of competitor's direction of 
effort 

(5) Other (specify) 

Serial No. C omment 

004 1, 2, 3» 4 play a part and most cause reprogramming 

R &. D . 

019 la order 1, 3 + 4 of above 

161 combination of 1 & 3 

237 (3) 80%, (1) 20% 

520 3 & 4 

Question 6: 

How are most personnel assigned to R Sc D projects? 

(1) Hired and trained especially for the specific R & D effort 

(2) Assigned as a result of past performance in other area 

(3) Assigned from permanent R St D staff 

(4) Hired to acquire specific technological capability 

(5) Other (specify) 



Comment 

varied 

combination of 2, 3 & 4 
varies greatly - all the above 
Combination of above 

Question 7: 



Serial No. 
019 
076 
131 
542 



What percentage of 


your 


R & D 


effort is 


subcontracted 


out? 




Less 


than 


5% 5-25% 


26-50% 


51-75% 


76-100?, 


Basic research 




(1) 


(2) 


(3) 


(4) 


(5) 


Applied research 




(1) 


(2) 


(3) 


(4) 


(5) 


Development 




(1) 


(2) 


(3) 


(4) 


(5) 



Serial No. 
123 
227 
497 
542 



Comment 

NA (for basic research) 

(0) Much of our development is done jointly with mfgrs. 
None or N/A 

(0) Info is not avail to me. 



Questions 8 and 9: 

Questions 8 and 9 as».ed for percentages and trends of funding 
from external government or commercial sources to support basic 
research, applied research or development effort. Comments received 
were generalized anu usually applied to both questions. 
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Comment 



Serial No. 

029 No Gov't Support 

066 (1) No outside funding 

112 don't do (gov't research, basic Sc applied) 

123 (indicated "NA" for all basic res. cats.) 

177 none 

225 Experience too limited to answer 

227 Much of our development is done jointly (answered 0) 

457 Not applicable 

497 None or N/A 

520 none (except commercial development) 

542 Best guess answers 

Question 10: 

What technique do you consider the most important in evaluating 
the return on investment or value of your overall R & D program? 

(1) Number of commercial products developed 

(2) Number of published technical articles 

(3) Number of patents issued 

(4) Cost savings estimates 

(5) New process developments 

(6) Market share trend 

(7) Other (specify) 

Serial No. Comment 

003 Sales of new products + 5x cost savings before tax + 

5x royalties 

004 combination of 1 & 6. 

019 The $ value of the successful projects for the operative 

groups. 

021 Earnings record of new products 8c processes 

022 resulting contribution to profits 

024 #4 & 5 

029 The number of quality commercial products which are 

being marketed successfully 8c profitably 
044 Application of results 

066 number of projects that are transferred to a production 

product or process. We are designated as an R 8c D 
department however we primarily do product, process, 
and mfg. devel. and test work for the various mfg. 
divisions . 

076 use of information or process by co . or customers 

within 5-10 yrs. 

084 Value of commercialized projects 

091 Profits 

097 Profit from products developed 

113 We do no basic research 

131 Profitability of products developed 

136 Effect of new projects on earnings. 

155 (1) 8c 5 

lol sales 8c net income resulting from R Sc D efforts 

167 projected profit from the programs 

178 Profit impact of new products plus product 8c process 
modifications. 
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question 10 (continued): 



Serial No. 
182 
192 

213 

227 



237 

278 

290 

30U 

305 

315 

323 

343 

351 

359 

360 

380 

423 

451 

472 

478 

520 

542 

544 



570 

586 



Additional 

Serial No, 
024 



325 



Comment 

Probable value to probable cost ratio 

tfe are not an R & D company, you should improve jour 

mailing list. 

profits generated 

The company engages in virtually no research. The 
development phases are conducted jointly by the company 
and suppliers of products that we provide (via leasing) 
to our customers. 

Change in return on investment 

Whether technology needed for efficient company oper- 
ations has been provided. 

Profit generated 

(1), 2, 3» 5 

demand from company’s profit centers for our technical 
efforts and participation* 

Profits generated traceable to R & D 

profits generated as a result of R & D programs 

No real good technique ever found 

Satisfaction as stated by the R & D sponsor which may 
be any of the above, but generally (1), (4r) , (5)> (6). 
Profit on long term basis 

Benefits to corporation established and verified by 
accounting Department 
New program wins 

Return of funds expended, bottom line profitability 
k and 5 

Estimated present worth &. probability of success. 

New market development 

(1) 1 St 5 & 4- about equal depending on R & D group 

1, 4 & 5 

Our group is small. 13 professional employees, 7 nun- 
professional, k clerical, 1 janitor. The retail food 
products activity in most food companies has most of the 
R &l D funds* This ranges generally from 0.1;o to 1 . 0/o 
of sales* Ours is G.l?o* 
estimate of value activities. 

This response does not include any of our government 
R St D contract which exceeds the corporately sponsored 
by almost an order of magnitude 

General Comments: 



C amine nt 

The R St D work among the electric utilities, especially 
that conducted by the Electric Power Research Institute 
constitutes a special case but does not generally fall 
within much of the above categorization. 

Questions have been answered in terms of the only R St D 
laboratory facility in the company* However, the total 
R Sc D effort of the company is several times larger than 
the laboratory* s. 
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Additional General Comments (continued): 



Serial No. Comment 

5b9 A reasonably good questionnaire. Might consider relat- 

ing R & D level to degree of maturity of each technical 
area, or to rate of growth of market. In some questions 
such as #1 and #6 of last section, an ordering would 
yield more information than selecting a single item. 

Additional appropriate comments from company letters forwarded 
with returned questionnaires: 

...Almost everything we do is on a first time basis and 
therefore there is no trend. 

(We) Use outside consultants with technological expertise to 
implement DEVELOPMENT to reduce operating costs of processes. 

With respect to Section Three of the form, where you have chosen 
words such as "primarily, highest, most," etc. you may have 
obtained a truer picture if you had invited weighting among the 
top 3* Decisions are usually made on the basis of several 
factors . 
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APPENDIX B 



NAVAL POSTGRADUATE SCHOOL 

MONTEREY, CALIFORNIA - 93940 



IN REPLY REFER TO 

Code 55 
0001-2 



Dear Sir: 

The Naval Postgraduate School has initiated a study 
effort to develop an improved decision model, technique, 
or criterion for project selection and evaluation which 
may be made applicable to United States Navy research and 
development planning. The questionnaire forwarded herewith 
is designed to survey general management and decision 
making techniques practiced by industry as guides to devel- 
opment of factors for Navy use. 

As your organization is a prominent leader in American 
industry, your inputs to this survey are ardently sought, 
and will contribute very substantially to the validity and 
utility of any assessments forthcoming. Early return of 
the completed questionnaire in the return envelope provided 
will be greatly appreciated. 

All information provided will be maintained as confi- 
dential; only aggregate numbers, statistical factors and 
trend analysis will be released as analysis information. 

Thank you for your valuable assistance. 



Yours, truly, 



SURVEY OF RESEARCH AND DEVELOPMENT MANAGEMENT 



General Instructions 

Definitions of basic and applied research seem not to differ significantly 
among the Department of Defense, the National Science Foundation, academic in- 
stitutions and industry. Development activities are, however, on occasion organi- 
zationally segmented into development, testing, evaluation, laboratory, design and 
other categories. Accordingly, for purposes of consistency in response to this 
questionnaire, the following definitions are provided to assist in interpre- 
tation of questions and potential comparison with other studies. 

BASIC RESEARCH includes original investigations for the advancement of 
scientific knowledge that do not have specific commercial objectives, although 
such investigations may be in the field of present or potential interest of 
the reporting company. 

APPLIED RESEARCH includes investigations directed to the discovery of 
new scientific knowledge that has specific commercial objectives with respect 
to products or processes . . . applied research differs from . . basic re- 
search chiefly in terms of the objectives of the reporting company. 

DEVELOPMENT includes technical activities of a nonroutine nature con- 
cerned with translating research findings or other scientific knowledge into 
products or processes. Development does not include routing technical services 
or other activities excluded from research and development. 

In order to provide a more consistent point of reference for your answers, 
we suggest that you scan the entire questionnaire prior to answering any of the 
questions. In your replies to this questionnaire, please treat your research 
and development facilities as a single group or entity. 

Please enter the most appropriate answer in the box at the right of each 
question number for the appropriate box or boxes. If the question calls for a 
response of more than a one digit response please place "ONLY" one digit per 
box. Disregard the numbers to the right of the boxes. 

SECTION ONE - BACKGROUND INFORMATION 

1. How many research and/or development laboratories or facilities 
does your company operate? 

(1) one, (2) 2 to 5, (3) 6 to 10, (A) 11 to 15, (5) over 15 



2. Your company size as indicated by total number of employees: 

(1) Less than 5,000 (3) 10,000 to 19,999 (5) 40,000 to 79,999 

(2) 5,000 to 9,999 (4) 20,000 to 39,999 (6) 80,000 or more 



3. As a very general industry classif cation , the greater portion 
o^ your company^ activities would ba: 

(1) Durable goods industry (3) Services industry 

(2) Non-durable goods industry (4) Diversified industry 3. 



4. (Optional) Please specify your official title in the company 
organization . 



1 

2 

3 

4 
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SECTION TWO - BUDGETARY CONSIDERATIONS 



1. How do you perceive your company's R i D effort over the past 
five years? 

(1) No significant change (3) Decreased 'signi Eicantly 

(2) Increased significantly (4) Suspended R & D efforts 

2. What percentage of your total operating budget does your R & D 
effort constitute? 

(1) Less than 2% (3) 5 to 7% (5) Greater than 10% 

(2) 2 to 4% (4) 8 to 10% 

3. Annual R & D budgetary level is planned as: 

(1) A percentage of net sales 

(2) A percentage of net profit before taxes 

(3) A percentage of net profit after taxes 

(4) A percentage of total operating budget 

(5) Specified other corporate financial index 

(6) Limited to active contracts; i.e., no in-house development 

(7) Not related to a financial index or factor of corporate 

operations. General basis is . 

4. Total R & D budget is limited to: 

(1) Not to exceed 2% of the above indicated financial index 

(2) Not to exceed 4% of the above indicated financial index 

(3) Not to exceed 7% of the above indicated financial index 

(4) Not to exceed 10% of the above indicated financial index 

(5) Other 

5. What percentage of R & D effort is directed toward basic research? 

(1) Less than 5% (3) 10 to 14% (5) 20% or greater 

(2) 5 to 9% (4) 15 to 19% 

6. What percentage of R & D effort is directed tov/ard applied research? 

(1) Less than 10% (3) 20 to 29% (5) 40 to 49% 

(2) 10 to 19% (4) 30 to 39% .(6) 50% or greater 

7. What percentage of R & D effort is directed toward develop- 
ment projects? 

(1) Less than 20% (3) 40 to 59% (5) 80 to 100% 

(2) 20 to 39% (4) 60 to 79% 

8. What percentage of basic research projects yield financially 
* profitable results withing five years of initial research? 

(1) Less than 2% (3) 5 to 7% (5) Greater then 10% 

(2) 2 to 4% (4) 8 to 10% 

9. Of those basic and applied research efforts that lead to a decision 
to go ahead with development, the majority of them involve a time 
period from initiation to development decision of: 

(1) Less than 2 years (3) 5 to 10 years (5) More than 20 years 

(2) 2 to 4 years (4) 11 to 20 years 

10. Of those R & D projects that lead to useful or profitable pro- 
ducts, the majority of them involve a time period from initiation 
to profit of : 

(1) Less than 2 years (3) 5 to 10 years (5) More than 20 years 

(2) 2 to 4 years ('•) 11 to 20 years 

11. What percentage of your basic research effort is internally funded? 

(1) Less than 20% (3) 40 to 59% (5) 30 to 100% 

(2) 20 to 39% . (4) 60 to 79% 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 
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12 . 
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12. What percentage of your applied research effort is internally funded? 

(1) Less than 20% (3) 40 to 59% (5) 80 to 100% 

(2) 20 to 39% (4) 60 to 79% 

13. What percentage of your development effort is internally funded? 

(1) Less than 20% (3) 40 to 59% (5) 80 to 100% 

(2) 20 to 39% (4) 60 to 79% 

SECTION THREE - GENERAL MANAGEMENT 

1. Selection and assignment of a specific R & D task to an organi- 
zational group or individual is based primarily on: 

(1) Availability of demonstrated project management expertise 

(2) Availability of technological expertise in-house 

(3) Formation of a new group incorporating both management and 
technological expertise (includes organizational reassignment) 

(4) Availability of support personnel 

(5) If capability not existant in-house, contract to best qualified 
management sub-contractor 

(6) If capability not existant in-house, contract to best qualified 
technologically qualified sub-contractor. 

(7) Available in-h^use personnel with best background adaptable 
with minimum training 

(8) Other 



13. 
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5. 



Which R & D project objective would get highest priority at your company? 

(1) Reducing operating costs of processes 

(2) Timeliness (with respect to ) 

(3) Improving company's competitive position by providing better 
products 

(4) Probability of success of R & D project 

(5) Soundness of technological approach 

(6) Estimated cost 

(7) Product diversification for company growth 

(8) Other (specify) . 2 
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At what stage in the budget formulation process is the funding 
level committed for R & D? 

(1) Beginning of the planning phase 

(2) When sales estimates are made 

(3) When profit estimates are made 

(4) At end of budget allocation, providing funds are available 

(5) Other (specify) 

If you anticipate equipment reauirements for R 6 D effort in addition 
to current R 6 D capability, your most common acquisition method is: 

(1) Purchase to establish or maintain general research capability 

(2) Purchase for specific R & D task 

(3) Transfer to R & D upon completion of other usage 

(4) Develope in-house as need arises 

(5) Other (specify) 

What is the most important criteria of your policy regarding 
reprogramming of R & D funds? 

(1) As new opportunities or ideas are generated 

(2) Upon failure of existing R & D projects to materialize useful 
results /products 

(3) Based on periodic review of all ongoing efforts 

(4) Observed/predicted major shift of competitor's direction of effort 

(5) Other (specify) 
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6. How are most personnel assigned to R & D projects? 

(1) Hired and trained especially for the specific R & D effort 

(2) Assigned as a result of past performance in other area 

(3) Assigned from permanent R & D staff 

(4) Hired to acquire specific technological capability 

(5) Other (specify) 
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What percentage of your R & D effort is subcontracted out? 



Less 


than 


5% 


5-25% 


26-50% 


51-75% 


76-100% 


Basic research 


(i) 




(2) 


(3) 


(4) 


(5) 


Applied research 


a) 




(2) 


(3) 


(4) 


(5) 


Development 


(i) 




(2) 


(3) 


(4) 


(5) 


What percentage of total 
is externally funded? 


R & 


D e 


f fort 


performed by your 


company 


Less 


than 


5% 


5-25% 


26-50% 


51-75% 


76-100% 


Government basic 
research 


(1) 




(2) 


(3) 


(4) 


(5) 


Government applied 
research 


(1) 




(2) 


(3) 


(4) 


(5) 


Government development 
Other commercial 


(1) 




(2) 


(3) 


(4) 


(5) 


basic research 
Other commercial 


(1) 




(2) 


(3) 


(4) 


(5) 


applied research 
Other commercial 


(1) 




(2) 


(3) 


(4) 


(5) 


development 


(1) 




(2) 


(3) 


(4) 


(5) 



Of those efforts indicated in question 8 above, the following 
past five year trends have occurred: 



Increased Decreased Varied Stable 



Government basic research 


(i) 


(2) 


(3) 


(4) 


Government applied research 


(i) 


(2) 


(3) 


(4) 


Government development 


(i) 


(2) 


(3) 


(4) 


Other commercial basic research 


(i) ' 


(2) 


(3) 


(4) 


Other commercial applied research 


(D 


(2) 


(3) 


(4) 


Other commercial development 


(D 


(2) 


(3) 


(4) 


What technique do you consider the 


most 


important in 


evaluating 



the return on investment or value of your overall R & D program? 

(1) Number of commercial products developed 

(2) Number of published technical articles 

(3) Number of patents issued 

(4) Cost savings estimates 

(5) New process developments 

(6) Market share trend 

(7) Other (specify) 



Additional comments are sincerely invited. 
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THANK YOU FOR YOUR ASSISTANCE. 
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V A R 0 1 1 R AND D PERCENT "DEVELOPMENT 
CODE 



(MC C C vO 
•-* O O I s - 
O' KIO C O' 



Ki =3- r- tP 

ip 

ir 



•— O 

• o 



























Ui 






c 




























u 






f— 


























3 


































«t 


•< 


UJ 






























i— * 


M 


o 






























(Z 


rr 


3 


X 




























UJ 




<x 


3 






















*-* 


o 




3: 


;> 


or 


CO 
























CC 




























*-> 


































ru 


































in 










CP 


CO 




o 


rvi 


o 
























a 






c 


o 


























c 




-o 


o 


>c 




















H 


G 


































>o 






•-* 


i 


IP 


pp 












o 


rf-"k 


w 
































I s - 






















to 












PO 


* 




















UJ 














* 




















CO 














* 




















<t 














-* 




















c 










x_s 




-ft 














to 




• 














w 


-ft 










a: 


> 


to 


X 




cs 










* 




* 




»H 


G 




£X 


OJ 


UJ 


— • 


• 


3 










-It 


-It 


M 






a 




Ui 


e 


2 : 


X 


CJ 


M 










* 


* 


-ft 














2 


f-r 




CO 










* 


* 


-K 










C’ 


G 


UJ 


X 


IP 


to 










* 


* 


-It 










f- 




yc 


<2 


O' 


►-» 












-* 


* 










CO 


co 


to 


X 


• 


X 










* 


* 


-It 


























«-4 




* 


* 


-ft 






















I s - 








* 


-ft 


-ft 






























* 


•It 


-ft 
























o 




O' 


* O" 


* O' 


* 




»— • 


o 


















fU 




po 


* Lr. 


-K r— 


-it 


<A 




rv, 
























-fr 


•ft 


-ft 


3 






>- 


a 


G 




G 




r- 




ac 




C 


* c 


* C 


-*t 


3 








CC 


G 


>o 


G 


c 






UJ 


* 


>- 


* p- 


* H 


-ft 


a 






3 




G 


K* 


G 


pp 






C: 


« 




-ft 


* 


•ft 








UJ 














* 


z 


* 


o 


-* o 


* G 


•ft 


o 






: 


pp 


CP 


1 


»■* 


o 




-It 


3 


* 


ru 


* ^2 


'It X 


-ft 


03 






G 










r^> 




•If 




-ft 




-K 


-ft 


-ft 








UJ 














* 




♦ 




•ft 


-ft 


-ft 








□L 














« 


•— • 


•-* -ft 


*— 




-ft •—> 


»-< -ft 


H* H 


f— • 


G 


U_ 












to 


































UJ 


o 




c 




o 


C 


o 




















CO 


c 




o 




o 


G 


G 














CO 




f— 


< 


• 




■ 




• 


• 




















c 


u 


<•-4 




ru 




P*> 


C7 


IP 














to 


3 


a. 






























<r 


X 




c 
























— 7 


aJ 


i— 


►—* 


• 


•— 
























•«* 


a 


cr 




> 


; 
























UJ 


Ca 


3 


•— » 


• 


< 
























3 : 






X 


o 


:> 



90 



VAR012 PERCENT PROP I T A BLF-F T VF YEARS 

cone 



in^co^ 
m r- c o nj 
rvj ru c c ev; 



— » ru ac ru 

fO 

fv 



ro 

CO 



-* 

* 

* 

* 

* 

* 

♦ 

* 

* 

* 

«* 

* 

* 

-* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

•* 

* 

* 

-fc 



in 

ru 



-* 




























•* 












G 








-If 








* 












«-• 








-It 








* 




















-If 








-* 


rvj 














w 




* 


•-4 


G 




-* 














o 




G 


* 




fVJ 




4C 










I— 




V* 


-It 


«-» 


-If 






>- 


4C 


or 














-It 




-If 








* 


Uj 




f~~. 




C 




C 


-If 


rr 


-If 








* 


c 




»— 




l- 


-K 


i — 


-If 


LU 


-If 






UJ 


-It 


2: 


v_^ 




-* 




* 




-* 


> 


-If 








4L 


22 




rvj 


* 


IT 


■* 


cC 


-If 


C 


-I* 






c* 


-K 




-* 




-It 




♦ 




-It 




-If 






tu 


-K 




* 




41 




-At 




-If 




-If 






ct 


* 


•-* 


*-« * 


M 


*-* * 


M 


-it 


•-4 


•-» -If 


M *-H 


-K *-« 




o 


u. 


O 




O 




C 




G 




G 




O «-» 








o 




o 




C 




O 




G 




O 








• 




• 




• 




• 




• 




2 








^■4 




ru 




fO 




cr 




in 




H* 









G 

O 



o 

<c 



c 

nC 



c 



m 

co 



O 



u 

n 2 

< < lL 
»-l M C 

£2 Ct 2 X 

LL < < w 

x > cr co 



N CT LP O C 

ro o it o <r 
■■««* cn «-* o 'C 

in 



CO • 

1 > tc I W 

(r uitt 3 • 

HJ C Z I t 

3 T ►-« 

o c ui >c tr* 

i— i— it-< O' 

co co <o 2: - 



•— • c O 

tror-or- 
a c ^ c r^. 

«-*«-> i 



co »- 

u 

w i: cl 

C X 

2 uJ I- • 

< c a 2 > 

LO O 3 *-< ■ 

X 21 ^ Z U 



lf> 

rvj 



91 



VALID CASES 122 MISSING CASES 



VAR013 DEVELOPMENT RO AHEAD - YEARS 
CODE 



ir m o o (r 

^ 43 C C A, 





















rvi 






IT. 


ru 




























G 






























tn 




















g 






























0 


































































UJ 






O 






























i— 






















X 


X 




























<r 


<x 


u: 


























*— « 


1— » 15 


























C 


CL 




x 
























UJ 


<z 


*< 


3 










Ki 










0 




X 


> 


a: 


to 










-O 










fC 
































































X 


0 


0 


0 


r^ 


in 






* 














X 3 


a 


<c 


0 










♦ 


*—* 












O 




G 


0 


0 








* 


c? 






►— * 


c 




















* 


cr 








X 3 










cr 


ru 








* 






























* 
























in 






* 
























Ui 


0 ~\ 




* 
























CO 






* 
























-«* 


KT 




-* 
























C-i 






-* 


-it 
















CO 




m 








-K 


-* 












Cfc 


> 


to 


X 




e: 






* 


* 








G 




cr 


UJ 


u* 


3 


• 


X 






* 


-If 








Cl 




UJ 


0 


X 


T 


O 


H-' 






-* 


-If 
















3 


*-* 




CO 






* 


-If 












e 


c 


UJ 


X 


in 


CO 


+ 




* 


* 












i~ 


»- 






a 


*-* 


-* 




-K 


-•c 












to 


C /3 


CO 




■ 


X 


* 




* 


-It 
























-K 




* 


* 
























* 




-* 


* 




rs 




















* 




-* 


-If 




in 
























* 


* 


d G 




*—• 


O 
















«* 


ru 


-If 


-If O 


ru 






fU 
















* 




-If CJ 


■* «-♦ 










>- 


tc 


G 


a- 


G 


<r 


ru 


* 


a: 


* 


-If 


C 








U 




O 




O 


;r. 


cr 


* 


U- 


-if c 


* c 










X 




C 




G 


r- 


»-• 


-fc 


o 


-It 1- 


-It H- 










LzJ 














* 


2: 


-It 


-r 


X— «r-r 








■ 


ru 


ru 


1 




X 




* 


! 


-If f\J 


-n in 




-it 






O 










K» 




* 




-it 




-K 


-it 






Ui 














* 




-* 


•it 


•If 


-r 






tr 














-* 




»— 4 f •-« 




H< < ►« M 


-it ^ 


M M 


G 


Ll. 












CO 






























Uj 


g 




C 


c 


G 


0 ^ 


















in 


o 




O 


G 


O 


O 












00 




fc- 


<t 


« 




• 


• 


• 


3 












•—r 


X 


Ci 


c 


<—• 




ru 


r-o 




*—4 












C/3 


3 


a 














to 












c 


X 




e 












to 








X 


Ui 


1 — 


*— t 


• 


►*4 




















*< 


c 


ot 


X 


> 


> 












X 








Ui 


C 


3 


»— » 


• 


-< 












w* 








X 


X 


v: 


X 


CJ 


> 



92 



VA9014 LEAD TIME TO PROFIT - YEARS 
CODE 



CC O O fV 

^ <— » o o tn 

xx cc ^ 



f V K1 Pt 

Kt 

*-r O 

















• o 






























• 






Ui 






C 


















• 






O 






#— 












*— * 






• 




2 


2 


















pj 






• 




-< 


-«l 


Ui 
















r*- 






• 




M 


M 


d 






















• 




o 


Ct 


2 


X 




















• 




u- 


-a 


-< 


2 




















M O 




x 


:> 


a 


CO 




















• cO 
























w< 






• 
























4 T 






• 
























* 






• 
























* 






• 




















*-\ 




- 1 C 






• 




ip 


rP 


o 


o 


in 


ru 










* 






• 




sO 


ao 


-c 


o 


r- 








^r 




-•c 






• 




c 




fp 


o 


*n 












* 






M C 
























* 






• -o 








ft 


^7 


Pi 












-* 






• 
























-It 






• 














CO 










-* 






• 














Ui 










-it 






• 














to 






-* 




-It 






• 














<T 






* 




* 






• 














u 






♦ 




* 






• 








CP 




• 








-K 




* 






• 




ct 


> 


CP 


X 


M 


cs 






♦ 




* 






*-* c 




ct 


u 


Ui 


3 


• 


2 






-* 




-* 






• Z 2 




UI 


o 


2 


X 


LJ 


t— » 






-* 




* 






m 








3 : 


►— 




CO 


r^ 




* 




-It 






• 




d 


o 


£jJ 


X 


IP 


to 


W-* 




« 




•* 






m 




1 — 


i— 




«*. 


cr 


•— 1 






* 




- 1 C 






• 




IP 


ip 


co 


X 


• 


X 






-fc 




-IC 


*■> 




• 




















* 




-* 


O' 




• 




















* 




* 






• 




















* 




* 






• 




















- 1 C 




-ic 


o 


*— ■» 


»— C 
















•* 


tv* 


- 1 C 




* C 


ru 


Aa 


• Pi 
















* 




4 C 




-It •— » 






• 


>- 


fp 


o 


IP 


c 


o 


IP 


* 


a 


-It 




-IC 


^ CT 




• 


tJ 


c 


c 


O 


o 


r- 


Cl 


* 


Ui 


* 


C 


-IC c 


l— 




• 


X 


un 


o 


^1 


C 


Pi 




* 


d. 


* 


*- 


-It l— 


* 




• 


ui 














* 


I!; 


* 




-If 


-It 




• 


3 


pi 


m 


ft 








-IC 


j 


* 


OJ 


-ic IP 


-* — - 




• 


O 










rP 




* 




* 




* 


-* 




• 


Ui 














* 




- 1 C 




* 


-It 


-* 


• 


a: 














* 


•— * 


•-* * 


•— « 


H * M 


*-* -It ^ •— » 


-IC •— 


T 

T 

0 


U_ 












co 






























Ui 


c 




cz 




o 


c 


c 
















CO 


o 




o 




o 


o 


H 










co 




i— 


*<5 


• 




• 




• 


• 


2 










*— 


^r- 


u 


o 






pj 




PO 




M 










CO 


2 


CL 
















CP 










c 


X 




e 














CP 






21 


U-> 


►- 


>-* 


• 


M 














t— f 






<c 


c 


cr 


2 


> 


3 














X 






Ui 


w 


3 




• 


< 




















X 


X 




X 


CJ 


> 



93 



VAR015 percent basic internally FUNDED 

CODE 



IT N C C 5 
O' ~ O O LO 
« tXj O C >C 










c 

o 

ru 



g 



tu c? 

U *- 

< < ^ 

W M c 
C(T2 Z 
< O 

x > cr co 



in 













o 




m 




G CC 




C 


















• 




ru cr 


o 


o 


ac 
















• 




W PO 


o 


G 


















*— O 


























• <v 






rc 


LP 


















• ^ 






» 
















* 




• 






















* 




• 






















* 




• 






















* 




• 






















« 




• 






to 




• 












* 




• 




cr > 


to 


X 


M 












* 




M o 




a: it* 


Hi 


~7*! 


• 












* 




• CC 




UJ c 


2 




L 












* 




• 






~y- 
















* 




• 




Q C 


Hi 


X 


in 












* 




• 




H 1— 


3C 


< 


O' 












* 




• 




<o <o 


CO 


X 


• 












* 


*— i 


• 






















* 


o 


• 












tn 










* 


ru 


• 






















* 




• 














c 


O' 


<-> O' 


*-» a 


♦ 




*-4 G 














ru 


r\i m 


ki in 


<\j r» 


-1C G 


























* az 




• 


>- 


P- G 




G 






JV 


C 


c 


r— 


* 


w 


• 


c 


— < G 


CC 


G 






LL 


*- 


V- 


fc— 


* a 




• 


2 


C G 


— - 


G 


in 




C 








♦ Hi 


* 


• 


LU 










* 


2 


g 


o 


o 


* > 


-It 


• 


zz 


ir 


ru 


«-4 


G 


* 


— > 


w r\i 


w car 


w ^ 


* c* 


-K 


• 


CJ 








m 


* 










* 


* 


• 


Ui 










* 




* 


* 


* 


* 


« 


• 


CC 










* 




-it *~» ►-« 




M * »-« >H 


* ►-* 


M ^ M 


T 

I 

0 


u_ 










C 




o 


o 


o 


G 


O -~i 














o 




o 


o 


o 


C 


C3 








CO 




i— 


• 




• 


• 


• 


« 


~ 








»-» 


X 


u 






rvj 


m 


^3 


in 


»— » 








CO 


— ✓ 


CL. 














CO 








c 


X 
















CO 






2 : w 


*— 


M 


• 














(►— 






< c 


CC 


2 


> 














31 






u; c 


3 


H 


• 




















X X 


5C 


X 


CJ 



94 



VALID CASES 1?7 MISSING CASES 



V A HO 1 #» PERCENT APPLIED INTERNALLY FUNDED 
CODE 



LT X’ C O ^3 

O *43 c o r- 

(X c o c 



c 

o 

ru 



o 

-c 



3 ^ 3 

>C 



L u C 

U H 

2 2 
< < iu 
»—>»—* CD 

cazi 

iii< o 

2: > a <0 



ru 































•C 


ru 




O 


=3 
















«_• f 














cr 




r- 


O 


PC 






























0 


0 


> c 


C 


















♦ 










c 




























* 










ru 






*-• 


ru 


IT. 


















-it 


















1 




















4c 






































* 






































* 






































* 






































* 






































* 


















<c 




• 
















4c 














:r 


> 


to 


X 


















* 








t-H 


c 




cr 


UJ 


UlI 


CD 


• 
















4C 










ac 




LkJ 


0 


21 


X 


u 
















* 




















►-* 


















* 














O 


0 


tu 


X 


in 
















* 














V- 


1— 


r 


<j 


cr 
















* 














co 


cc 


<0 


X 


• 
















* 






































* 






































* 






































4c 
























cc 


O 


a 


CO 


a 


-0 


a* 


* 








*— 1 


O 
















ru 


*-> to 




IT 






4c 


0 








^3 




























4£ 


CO 










>- 


^3 


0 


ru 


0 


IT t 




cr 


c 




c 




O 


4t 












O 


O 


0 


0 


0 


ru 




Ll> 


h- 








1 - 


4c 


CO 










2: 


nO 


cr 


0 


0 


0 




c, 












4C 


cu 










oJ 












w 


2 


0 


*— • 


0 




0 


4C 


D> 












■C 


ir 


- 0 


w* 


ru 




* 


nj 




^7 




>c 


* 


c 










e 










ru 


* 






* 




* 




4* 












u. 












★ 






* 




4C 




* 




4C 








a: 












4c 




* »-• 


►-* * 


M 


>-« * 


*— 


•— 4C 




41 


•— ! >— 


M 


O 


u. 












C 




0 


0 




c 




O 




O 


4—\ 


















C 




0 


0 




0 




C 






CD 












to 




I— 


• 




• 


• 




• 




• 






X 














X 


CJ 






rvj 


K* 




C7 




in 


















<0 




CL 






















CO 












C 


X 
























CO 








2 : 


tu 


1 — 


•— * 


• 






















1— » 








-c 


0 


cc 


X 


> 






















3 : 








LU 


a 


3 


M 


• 






















w * 








X 


X 


5C 


X 


O 



95 



VALID CASES 1UU MISSING CASES 



VAR017 PERCFNT DEVELOPMENT INTERNALLY FUNDED 
CnDF 



















0 s 


#v 


o c 


C! 


















nj 


CO 


G C 




















cr 


CC 


o c 


CC> 


















=5 




o 
























GO 








































*— ► o 
























• o 
























• rv. 
























• 






UJ 




c 














• 






o 




1— 














• 




2 : 


X 


















• 




<* 


<T 


Uj; 
















• 




►— * 


t— ♦ 


O 
















• 




Ci 


CT 


X X 












*-> 




• 




UJ 


-a 


-a. X 












I s - 




M O 




X 


:> 


CC CO 












rvi 




• X 
























• 
























• 

• 
























• 




«/ 


cr 


nj o 


in 










* 




m 




I s - 


m 


m o 


po 










* 




m 




o 


cr 


^ o 


in 










-* 




M C 




















* 




• nj 








fO LP 












♦ 












S 












* 




• 




















•* 




• 




















* 




• 




















-* 




• 




















* 




• 




















* 




• 








CO 


• 










* 




• 




tr 


> 


CO X 


*-* 










* 




•— ' c 




a: 


UJ 


Ll< — r 


• 










* 




• CC 




Ui 


CL 


X X 


U 










* 




• 








3L i-4 












* 




• 




cl 


C 


Ui X 


in 










* 




• 






1— 


*1 C 


<r 










* 




• 




co 


CO 


CO X 


• 










-* 




• 




















+ 




• 




















* 




• 




















* 




• 












>c o 


o 


^ o 


-o cr 


* 




#— o 












nj 


m KT 


Kl in 


r- 


-r o 


fVi 






















* CC 




• 


>- 


g 


G 


^ G 


CC 


cr 


c 


O 


c 


-It 




• 


CJ 


cr 


g 


o 


OJ 


uJ 


•-* 




*- 


-* a: 




• 


JL 


>© 


o 


-C G 


G 


o 








« Ul 




• 


Ui 












O 


G 


^ c 


* > 




• 






m 


CC — 


G 




w ru 


^ cr 


X) 


* o 


v> 


• 


G- 










* 






* 


-* 




• 


Ua 










♦ 


* 


-it 


* 


* 


* 


• 


ql 










* »-H 




^ H l-t 




* ^ 


►-» -It •— • •-> 


•— • o 


Lk_ 










o 


c 


C 


c 


g 


C ^ 














c 


o 


o 


o 


o 


<x 










CO 


»— 


• 


• 


• 


• 


• 


x 










*-> X 


U 




nj 


fn 




in 


•— » 










CO X 


X 












CO 










c x 














CO 






2L 


Uu 


1— M 


• 












M 






<r 


c 


CC X 














X 






Ui 


c 


XJ *-» 


• 


















X 


X 


^ X 


CJ 



96 



VALID CASES lflS MTSSINfi CASES 



O' O C 0 CQ 


>c 


x in /i 


0 






?V O X C >c 




O Q* O' 


0 


ru 




c c in 0 ^ 




rvj 


0 






ro ru . lh 




ru 


ac 


no 




• >c 










CO 












UJ 




to 








CO 




UJ 








-< 


CO K 


CO 








CJ 


< 


to 




• 




X CJ 


c 


er > <0 


X 


t-H 


CJ 


<n z: cl 




a: u* ui 


; 


• 




C X 


e 


iii c z 


X 


O 


M 


Z Ui K m • 


*— * 


3: 


»~» 




<0 


<t x cr x > 


, f 


C O Ui X 


in 


to 


UJ Q 3 h • 


< 


► — » — XI 


c 


a 


»H 


r x y r u 


:> 


CO CO to 


X 


• 


X 

























O* 


















vC 




































-It 


















-It 


















-It 


















-It 


















-It 


















* 


















-It 


















-It 














z 




-It 














c 




-It 


















-It 














cr 




-It 














Ui 




k 














•— 




* 














M 




* 














£L 




k 














CJ 




-it 


















k 














f- 




-it 


O 












X 


9 — 


-it 


z 












Ul 


ct 


-k 


X 




X 








X 


— • UJ 


4 t 


tj 




33 


to 




t- 


z 


CL 


k 


Ui 




C 


X 




x 


CJ 


X 


k 


•— 




CL 


u> 




CJ 


M 


UJ 


-k 






CJ 


X 




X 


to 




-k 


CL 








^■s 




CO 




-k 


to 










Ui 






k 


x 






cr 




cr. 




•it 


-k 


c 






O 




H* 




-it i~ 


k 


X 


•%— ' 


X 


a 




to 


cc 


* X 


-k 


9 




or 


X 




1 - 


-<* 


* O 


k 


x 


k 


O 


— ; 




3 D 


X 


-It x 


-k 


•— * 


k 


lL 


w CO 


w 


C 


M 


-It 


-k 




k 










a: 


k 


-k 




k 




k 


k 




CL 


»— k •-* 


-k 


*-< 


•-* k 


•— « 


•-* k •— 1 


»-* k 


•— » 


UJ 


0 


0 




0 




0 


0 




0 


0 


0 




0 




0 


0 




C 


















CO 1 J 




ru 




ro 






in 





o 

or 

> 



irs>coor^ 
in ru c c r** 
f\i c c a 



t\j n a 
LO 
i r 



Ui c 

U I- 

Z Z 
< < UJ 

m m c: 

C; cr z 2: 
ui -< <s zr 
x > or cc 



*-f o 

• o 



•— c 

• GC 



•-» c 
• us 

*—\ • 

h» • 

Kl • 



M O 

* ■ O 

* • 

-J * • 

CT * ^ • 

Z k *-* • 

^ X * • 

U * X ^ • 

u; k *— cc • 

»- k • 

k LkJ *-> O 

lu k co • ru 

c: ^ X 

^ k CT Or • 

tO * JL w u, • 

* K * f X 

* X * 2: -k •— • 

k C k *“*> He C w • 

* k k • 

k k k k • 

»-« k k ^ * k k ^ »-i c 

o c c c 

OOO o 



>c co •-• 

co 

to 

M 

X 
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FREQUENCY 



VAR0t9 HIGHEST PRIORITY CRITERION 
CODE 



cn c o c c 
o o ru 
o in o o oc 



c 

o 



in 

ru 



* 

-pc 

* 

-pc 

-K 

•pr 

* 

* 

* 

* 

-* 

* 

* 

* 

* 

♦ 

* 

* 

* 

* 

* 

* 



O 

<r 



o 

>o 



o 









-Pc 




GO 










<n 


-pr 




in 




X 




GO 


go 


-PC 


p— 


UJ 




p— 




P— 


Uj 


-It 


»-* 


LJ 


•» 




* 


cn 


o z 


-pr 


CO 


LJ 




o 


* 


c 


•— » 


-PC 


cr 


Z- 




cr 


-It 


u 


-i 


-Pt 


CL 


(/) 




LZ 


M 






-pc 










-pc 






-pc 










* 


LU 




-PC 


P— 




» 




-pc 


U 


^ i 


* 


tu 






<z 


* 


3 


&u 


-PC 


CL 


cc 


-PC 


CL 


-pc 


Cl 


* X 


-pr 


X 


o 


-pt 




* 


UJ 


* •-* 


•Pc 


X 


^ ar 


-PC 


O 


* 


a: 


-pc P— 


-It 


LJ 


Ol 


-It 


LJ 


* 




-Pt 


-it 




* 


•PC 




* 




-pc 


-It 




-PC 


-PC 




-it 


#— « 


»-4 -|t *-♦ 


*- -it 


•— i 


-PC 


-PC 


►i 


C 




o 


C 




c 


c 




o 




c 


o 




o 


o 




• 




■ 


• 




« 


« 




<—* 




rvi 






CT 







r*j 



* 

* cr 

* U- 

-* x 

* »— 

* c 

* 

* 

* »-* 

o 

o 

« 

ao 



* 

* K-t 



C 

ir 



c 

rv* 

>- 

LJ 

2 

UJ 

C 

Ui 

a: 

o u. 



go 

cc 



nj =3 K* to 
c 
IT 



Lu O 

LJ I— 

2 X 

< < li. 

(Z 

o a: 7 r 

II < < D 

x :> a. <n 



(O C O cc f\j 

— • o ^ 



rv^tr 







U3 




• 


Ct 


> 


CO 


■«e_ 




a: 


tu 


tu 




• 


ai 


c: 


X 


X 


LJ 






•A. 


M 




C 




tu 


X 


tr» 


P— 


P- 


y 


<x 


O' 


if? 


CO 


GO 


X 


• 



eroccor^ 
X C IT C fU 

c c - c >c 



K' tO — • -r- 

>C 



m p- 

MTU 

co 7 a 
C X 

2 iiJ hn • 

*x c cr z > 

UiD2 w • 

r r y r u 
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VALID CASES t45 MISSING CASES 



VAR030 BUDGET COMMITTED 
CODF 



O' o c c: 
>o O O tO 
iO AJ C O LO 



























— » 


rv: 


=3 


O' 


rvi 
































ru 


































K1 
























o 


































o 


































'•’** 






Ul 






0 




























u 






»— 




























X 
































-«* 


-< 


Ui 






























•-4 


1— » 


e? 






























G 


ct 


2 


X 




























Ui 


-< 


<x 


X 
























o 




X 


> 


QC 


to 
























cC 
















































-* 


































-* 


































♦ 


































-* 


























o 


-O 


c 


cr 


-* 
























ru 


a 


r- 


o 


to 


-It 
























•-* 


53 


>o 


o 


o 


* 




















o 














* 




















-c 






»-< 




to. 


ru 


-* 


































* 


































* 


































* 


































* 


































-* 


































♦ 




























to 




• 


* 
























X 


:> 


to 


X 




* 






to 














o 




cr 


Ui 


U- 


, 


• 


* 






rv. 


















us 


c 


X 




o 


-* 












O 
















3: 






* 












«-« 












c 


o 


u. 


X 


io 


* 


Ui 


to 




-c 


















*- 


it 


<1 


o 


* 


CO 


■— * 




















to 


to 


to 


X 


* 


-* 


<f 
































-* 


X 


i- 


t— 




<t 
























* 


CL 


CO 


-* CO 




:> 
























* 




Ui 


-It UJ 




-a. 
























-* 


C3 




* 






-It 




fO 


#— * 


o 














* 


X 




-it 






-it 








r\ i 














-* 


t— 1 


-It 


-it *- 


-R 




-It 










>• 


to 


o 


c 


o 


<*-« 


* 


X 


-It CO 


-it 


-It 


CO 


-it 


rr 








CJ 


ac 


c 


=3 


o 


o 


-it 


X 


-It u> 


-it Lk. 


-It 


Cl 


-It 


UJ 








X 


rv 


o 


C 


o 


rv 


-K 




-1C _» 


-It c 


-It 


X 


-It 


X 








4jJ 












* 


' 


it <x 


* ct 


-It 




-It 


!~ 








— ; 


fV 








to 


M 


a_ 


* CO 


-it CL 


* 


u. 


-It 


c 








G 






> 




-c 


-* 




-It 


-H 


* 




-It 




-K 






Ui 












* 




-It 


-It 


-It 




-R 




-R 






cr 












-It 


*— • 


-It 


H * H 


»— -It 




►- -K 


*-4 


fc—t -R 1— • 


*-< 


o 


u 












O 




o 


c 


o 




O 




o 


















c 




o 


o 


o 




c 




o 












CO 




t— 


• 




• 


• 


• 




■ 




X 














X 


O 


—> 




<M 


ro 






in 




*— 1 












to 


3 


X 


















to 












c 


X 




















to 








X 


Ui 


*- 


*“> 


• 


















•— 1 








c 


c 


CL 


X 


> 


















2 : 








aJ 


C 


3 


*-♦ 


• 


























2 : 


X 




X 


t_J 
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valid cases missing cases 



VAR021 EQUIPMENT ACQUISITION MODE 
CODE 



N k: C O N 

co €\i o o cr 
r- o o o o 





























^r 


>o 


fV 






























r- 
































nj 




















>-4 


G 
































O 
































<•««» 






































UJ 






C 






IT 




















G 






H 
























2 


X 






























<l 




Ui 




























v— » 




L2 




























C 


CL 


Z 


X 


























UJ 






X 




















M 


O 




X 


Z> 


CL 


CO 






















a: 


















R 
































R 
































R 
































R 
































* 




























o 




* 


















in 


ru 


ro 


o 


f-n 


tn 




R 


















a: 




ro 


c 


>o 






* 


















o 


o 


CO 


o 


r- 






R 














o 


















R 
























IT 








R 
































R 
































* 




























* 




R 




























* 




* 




























R 




R 




























R 




R 






















CO 




• 


R 




* 


















CL 


> 


CO 


X 


M 


* 




* 












>-• 


G 




CL 


UI 


UJ 


X 


• 


* 




R 


















UI 


e 




X 


c 


R 


_J 


* 






Ui 


















t— 1 




R 




* 




X 


CO 












e 


c 


Ui 


X 


LP 


R 


CL 


R 


• 


IH 


X 












»- 




L 


<J 


CL 


R 


UJ 


* 


CJ 


X 


G 












CO 


CO 


to 


X 


• 


R 


-£ 


R 


Ui 


f- 


*-' X 






















* 


UI 


R 


G_ 


►-• 


O f 






















R 


LT 


* 


CO 


rr 


— X 






















R 




* 






*-% 


>c 


r-x 


















R 


U. 


R 


Ui 


CL 






^3 


t-~ 


G 














* 


CO 


R 


CO 


*Vi Ua 


CL 








<Vl 














R 


<* 


R 


<J 


Cl 


c 










>- 


G 


o 


d 


G 


O 


R 


X 


R 


X 


CO 


W i. 


a 








O 




o 


r\j 


G 


<— » 


R 


u 


R 


CJ 


2 


U 


lL 








X 


a 


o 


— 


G 




R 


a: 


R 


cl 


<t 


R > 


^ X 








iLi 












* 


r? 


R 


— r 


CL 


* UJ 


f— 


w 






X 




nj 


rvi 


— * 


f>Ji 


R 


Q. 


R 


CL 


w *— 


R C 


R X 








c? 










IT 


* 




R 






R 


R 


R 






Ui 












R 




R 




R 


R 


R 


R 






CL 












* 


fc— <» 


*-* R 


M 


R ►-* 


Ht R M 


M R M 


R M ^ 


»-* 


G 


Cl 












O 




G 




G 


G 


c 


O /■> 


















G 




C 




G 


C 


o 


e? 












CO 




Ir- 


• 




• 




• 


• 


• 


2: 












h- 


X 


U 






ru 




K> 




ir> 


M 












CO 


X 


JL 
















CO 












c 


X 


















Uj 










UI 


»— 


*—» 


• 
















M 








*t. 


G 


or 


X 


> 
















X 








Ui 


C 


G 


M 


• 
























X 


X 


yc 


2^ 


U 
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VALID CASES 103 MISSING CASES 



VAR022 RF. PRDGR A MM I NG CRITERION 

code 



a c o c f\j 

o o o o 

t^- o o c ir 























AJ 




^3 


UD 


Al 




























O' 






























K* 












CC 








►X C 






























• c 






























• 






ui 






o 
























CJ 






lh— 


















• 




X 


X 
















41 








• 








Ui 














4t 








9 




•— » 


#— * 


cr 














•k 








m 




c 


X 


X 


y 












4t 








m 




Ui 




-a 


3 












4c 








•“* C 




X 


> 


cr 


(O 












4c 








• CO 






















41 






























4c 








• 






















4t 








• 






















* 








• 






















* 








• 






















* 








• 




r- 


cr 


in 


O 


r- 










« 








• 




® 


cr 


cr 


o 




ir 
















• 




o 


o 


«-» 


o 


nj 










* 








H c 






















* 








• >o 






1 


t 


IT 


rvi 










* 








• 






















4t 








• 






















4c 








• 














w 








* 








• 






















* 








• 














* 








* 








• 














* 








4C 








• 








«r 




• 


* 








4C 








• 




X 


> 


cn 


X 


1—4 


* 








41 








c 




a 


u 


Ui 


— 


« 


* 








4c 








• ^ 




Ui 


o 


X 


X 


e 


4r 








4T 








« 








3 


*— 




* 






Ui 


4c 2L 








• 




CL 


c. 


Ui 


X 




* 






cr 


* Ui 


4- 






• 




\— 


4~ 


'sr 


*<. 


cr 


* 






"*"T 


4c t-f 


£T 






• 




in 


CO 


CO 


X 


• 


* 








4t ;> 


C 






• 














* 






•— * 


* Ui 


U. 






• 














4c 








4c cr 


u. 






• 














« 




4T-> 


LL 


4c 


Ui 






• 














* 




^3 




4c U 


40 




^r 


•-» c 














4i 


U, 




4- 


4c 








• rvi 














* 






u 


4t CL 


»~ 






• 


>- 


c 


o 


■•»«« 


<r 


>o 


4C 


*— 1 




<Jb 


-k a 


Ui 


CL 




• 


CJ 


f\j 


o 


oc 


o 


IT 


« 






-3 


-k •-« 


a. 


U! 




• 


JC- 


^r 


o 


>c 


o 




4r 


3: 




C 


4c rr 


x 


X 




• 


Ui 












* 


Ui 


w 


a: 


4C Ui 


w C 


^ H- 




• 


ZD 


nj 


ro 


ft 




Kl 


4t 


2 




CL 


* a 


CJ 


r*^ 




• 


c 










^T 


* 




4t 




41 


4C 


4c 


4t 


• 


u 












* 




41 




41 


4C 


4c 


41 


• 


cr 












* 


•-* 


t-» 4t 


9~ f—i 


4t ^ kh 


4c ■-• 


•-* 4t 9-* 


•-» 4T 9-* 


i 

T 

0 


u_ 












O 




O 




O 


O 


O 


C ^ 
















o 




o 




o 


c 


o 


<J3 










CO 




1— 


• 




■ 




■ 


• 


• 


zz 










•- 


X 


CJ 






ru 




f*> 




in 


*— 1 










CO 


23 


CL 
















CO 










c 


X 


















CO 






2 


Ui 


*— 


►-J 


• 
















•-* 








c 


cr 


X 


> 
















X 






Uu 


o 


3 


1— * 


• 






















X 


X 




X 


CJ 
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V A L 1 0 CASES 1 n 3 MISSING CASES 



VAH0?3 PFRSONNEL SELECTION 
CODE 



a - c c a: 
nj rn o o x 
cc r*- o o cc 



cc c ^ m 
x 

in 

CC K. c 

• o 



























UJ 






c 


























CJ 






1- 










*— <■ 














X 


X 






























<i 


-< 


u. 














ft 
















i-l 


o 














ft 














c 


X 


X 


X 












* 














UJ 


<L 


c 














-it 








•— < 


o 




X 


:> 


X 


03 












-it 










cc 






















♦ 
































* 
































* 
































* 
































ft 
































ft 














» * 


in 


ru 


o 












* 
















in 


cc 


o 












ft 














o 


tc 


o 


o 


>c 










-It 








•— 


o 






















-If 










>c 










in 












ft 
































* 
































ft 
































-it 




























ru 




-It 




























K1 




-it 
































•It 


















o: 




• 










ft 














X 


> 


03 


X 












-it 










c 




X 


UJ 


UJ 


3 


« 










-ir 














UJ 


o 


X 


?r 


CJ 






V-X 




-It 


i 
















X 












X 


-It 


C 












o 


o 


UJ 


X 


x. 




c: 


* 


a: 


* 














»- 


»- 


2ft 


< 


a 




IU 


* 


C 


It lx. 


X 












in 


03 


03 


X 


• 


nj 


Or 


•It 


II. 


ft il_ 


^ u 
























*— » 


ft 


X 


-It «* 


cr ui 
























X 


* 


UJ 


ft 1- 


i— 


























ft 


X 


ft CO 




>o 


y— \ 




















u 


* 




-It 


u 




t*1 


*— ♦ 


c 
















•— » 


ft 




-ft 


•— » 








ru 














v-* 


u_ 


ft 




-ft 


u. 










>- 


A- 


o 




o 


>c 




»—• 


ft 




-ft 


w *-l 


X 








u 


IT 


o 


PO 


o 


o 


-IT 


CJ 


* 


>- 


-ft X 
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VALID CASFS 144 MISSING CASES 



VAR024 PERCENT BASIC SUBCONTRACTED 
CODE 
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VALID CASES \1>U MTSSING CASES 



VAR0P5 PERCENT APPLIED SUBCONTRACTED 
CODE 
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VAR02e> PERCENT DEVELOPMENT CONTR ACTED 
CODE 
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VALID CASES 144 MISSING CASES 



VAR0?7 EXTERNALLY FUNDED GOVT BASIC 
CODE 
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VALID CASES 1 S3 MISSING CASES 



VAR028 EXTERNALLY FUNDED GOVT APPLIED 
CODE 
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valid CASES 1 34 MISSING CASES 



VAR029 EXTERNALLY PHNOfn GOVT DEVELCIPMENT 
CODE 
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VALTO CASES 136 MISSING CASES 



VAR030 FXTFRNALLY FUNDED COMM BASIC 
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k 




*-> 








H O 
















k 


in 


fV 


m 






















k 








c 




• 


>- 


m 


0 


po 


O 


5T 




k 


a: 




o 


c 




■ 


O 


cc 


0 




<z 


cr 




k 


UJ. 


in 


in 


<—« 


w 


• 


Z 




0 


to 


0 


in 




k 


Cl 


fM 


i 


f 




• 


uJ 














k 


~7* 


I 


x 


x 


k 


• 






<— • 


cc 




K1 




k 


3 


ip 


W fVi 


w f- 


k 


• 


e 










X 




k 










k 


• 


lAJ 














k 




k 


k 


k 


k 


• 


QC 














•-« k 


►- 


M + M 


*— k •-* 


►- k m 


H * Ht 


I 

I 

0 


(L 












to 




























JJ 


c 




c 


<zr 


c 


c ^ 
















cc 


c 




o 


o 


c 


w 










CO 




1— 


•< 


• 




• 


• 


• 


x 










»-» 


X 


CJ 


u 






rvi 


m 


in 


M 










CO 


3 


Ql 














in 










c 


X 




0 












in 






JZ. 


Lu 


H- 


»-» 


• 


*-* 












•— > 






<s 


a 


ct 


X 


> 














2 : 






uj 0 


3 


•-4 


• 


•a 












w 






X 


X 


!L 


X 


0 


> 
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VAR011 EXTERNALLY FUNDED COMM APPLIED 
CODE 



<c >c o c o* 

in o o o 

c n c c q 



d c ^ 

r^ 















• o 




























• o 




























• f\j 




























• 






Ui 






o 
















• 






o 






*- 
















• 




3 


3 






















m 




<x 
























m 




M 




o 




















m 




C; 


ct 


3 


X 


















m 




UJ 


««a 


-a 


3 


















O 




X 


> 


Ct 


to 


















• 4> 




























• 
















rvi 












• 




























« 

• 




























• 

• 




4) 




r- 


o 


O' 


nj 


w 












• 






tt> 


«-• 


o 


o 


^f-4 














■ 




c 




43 


c 






■It 












M o 
















* 












• Oi 








Kl 


in 






-K 












• 
















■k 












• 














to 


* 












• 














Ui 


-k 












• 














to 


-It 












• 
















-It 












• 














u 


* 












• 








(C 




• 




■It 












• 




ct 


> 


<n 


X. 


M 


o 


* 












M C 




*x 


tli 


IU 


3 


■ 


3 


■It 












■ cc 




Uj 


3 




X 


CJ 


•—* 


■It 












• 








3 


*— • 




to 


-It 












• 




c. 


C 


Uj 


X 


in 


to 


-k 












• 




k— 


1 — 


it 


<r 


ct 


*— i 


■It 












• 




to 


to 


to 


X 


• 


X 


* 












■ 
















-It 












• 
















* 












• 
















-It 








*-> 


rvi 


• 
















-It 








-43 


v- 


•-* o 
















-It 


tn 


in 








• d 
















•k 








C 




• 


>- 


cr 


c 


in 


c 


in 


in 


k 


CL 




C 


c 




• 


CJ 


in 


c 


in 


c 




KT 


-* 


Ll> 


tn 


in 






• 


2 


n- 


c 


o 


C 






-k 


C: 


f\i 


B 


a 


%-# 


m 


u; 














* 


Z 


K 


>o 


vC 




m 


3 






rv 


-*-■ 


cr 




-k 


3 


w in 


^ r\j 


r- 


-k 


• 


o 










43 




-k 








-k 


■k 


• 


UJ 














■k 




* 


■k 


-k 


■k 


• 


ct 














-k 




H * H 


•-**»-* 


»-H -k #-* 


M * M 


M •— * C 


Ll- 












to 




























UJ 


c 




o 


o 


o 


o 
















to 


c 




o 


o 


c 


es 










to 




*- 


< 


■ 




• 


• 


« 












#-* 


X 


C 


u 






nj 


pn 


in 












to 


3 


a 














cn 










c 


^r 




c 












1/3 








Ui 


I— 




• 


M 












*— * 






«* 


a 


□r 


3 


> 














2“ 






Ui 


U* 


3 




• 


< 












w 






X 


X 


it 


X 


u 


> 
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VAR032 EXTERNALLY FUNDED COMM DEVELOPMENT 
CODE 























a-f 


G 


O 


rvi 






















KT 


O 


o 


ru 




















o 


i— 


o 


G 


=ci 




















• 


• 


• 


• 


• 
























=3 


in 










































































Hi C 




























• o 




























• OJ 




























• 






bJ 






r> 
















• 






U 






H- 
















• 




2T 


X 






















• 




C 


•«* 


Iki 




















• 








O 




















• 




c. 


Q1 


X. 


X 


















• 




4J 




«< 


; 


















M O 




X 




oc 


co 


















• 43 














c 














• «-> 














CVJ 














• 




























• 

• 




























• 

• 




ro 


in 


^ 4 


o 


m 
















• 




r- 


in 


PO 


o 


m 
















• 




G 


CC 


PA 


G 




•ft 














*— G 














* 














• fU 








m 


in 




* 














m 














•ft 














m 














•It 














m 














■k 














• 














•It 














• 














* 














• 














-* 














• 








CD 




• 


•ft 














• 




ct 


> 


cn 


X 


»— » 


•ft 














o 




€t 


UJ 


uJ 


5 


• 


* 














• CC 




UJ 


c 


X 


X 


o 


•ft 














• 








-3. 


*— r 




•* 














• 




C 


c. 


UJ 


X 


in 


•ft 














• 




1— 


1— 




<J 


a 


* 














• 




C/3 


CO 


to 


X 


• 


•It 














a 














•If 














• 














•If 












rf—i 


* 














•If 












G 


• 














•If 




r- 




#— » 


* 1 


w* 


H-p O 














•It 


tn 






tn 


=3- 




• 














•If 










G 




• 




r- 


G 




G 


CC 


* 


tr 




o 


IT 


O 




• 


o 


r^ 


G 


ru 


G 


^3 


-* 


Ui 


tn 


ITS 


N 


— * 




• 


2 


ru 


O 


K' 


C 


a 


•If 


Q 


AJ 


1 


V 


1 




• 


UJ 












* 


X 


^ 1 


43 




4: 




• 


3 




«-< 


O 


— • 


nC 


■If 


3 


IT 




' LH 




•k 


• 


e 










4j 


* 




•It 








•k 


• 


LU 












-* 




•k 


* 


■k 


* 


•k 


• 


cr 












•It 


*-• 


•— * -k ^ 


* *-* 


•— *k •— » 






H- l-l C 


u_ 












C 




g 


O 


o 


G 


o 
















o 




G 


o 


o 


G 


03 










CO 




1- 


• 




• 


• 


• 


• 


2 












X 


03 


<— « 




rvi 


ro 


cr 


in 


1—4 










<n 


3 


CL 














cn 










o 


X 
















cn 






2L 


LL 


•— 


Hi 


• 














»-» 






-4 


c 


or 


X 


> 














X 






Ui 


C 


3 


*—s 


• 














n->> 






X 


X 




X 


CJ 
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VALID CASES 137 MISSING CASF.S 



VAR033 TREND RPVT BASIC 
CODE 



to o o x 

(C IT O O 

r*- — o o it 



ki — ki tr 



M C 













*—* 






O 


























a 
































r- 












u* 






c 


























u 






*— 
























2 


X 






























<1 


-a 


Ui 




























M 


•— # 


tr 


















■w 










e 


a: 


zz 


X 


























UJ 


-«* 


•< 


3 
















* 




>— * 


o 




X 


> 


ct 


to 
















-* 






CO 


























It 
































* 
































* 
































* 
































* 
































-If 










«-» 




,N- 


c 


X 














-It 










o 


r^ 




<2 




mi 












•ic 












o 


<— * 


O 


— • 














•It 




M 


o 


























-It 






x 












fO 














•K 














1 


















-* 




















cn 












-It 


rf— * 


















Ui 












•If 


O’ 


















to 












* 


ro 


















-a 












-R 




















i-J 












* 














to 




• 














•It 










ct 


> 


CO 


X 


*— 4 


O 












-It 




*-< 


c 




cc 


u 


u- 




■ 








^■N 






•It 










UJ 


c 


~z 


X 


o 


*—* 






o 






-If 














3: 


Ht 




<o 












* 


it 








c 


c* 


Ui 


X 


ir« 


to 












* 


it 








1— 


1- 


:* 


<1 


O" 


•-I 


<—s 










-it 


IC 








CO 


CO 


CO 


X 


• 


X 












•it 


it 






























-it 


n 






























-it 


It 






























it 


It 






















U- 


* 


UJ 


o 


it 


It 




o 


















t/5 


•* 


to 




it 


It 




ru 


















<1. 


♦ 


<t. 


c 


It u. 


If 






>- 


X 


c 


CO 


o 


m 


rr 




U 


« 


Ui 


UJ 


It _J 


H 






u 


Ml 


c 


X 


c 


o 


<■-* 


-* 


ct 


* 


OL 


M 


it a: 


It 








Ml 


c 


<\J 


c 


*4 




•* 


u 


-* 


1-5 


<r 


it <* 


It 






u.* 














* 


X 


* 


Le- 


w ^ 


it i- 


It 






— I 


Kl 


CJ 


9 




tv 




-* 


M 


♦ 


er 


> 


ir to 


It 






O' 










M* 




-* 




-* 




* 


If 


It 






UJ 














* 




** 




■K 


It 


It 






tr 














* 


*— 


•-* * 


•-< 




H * H 


•-» It *-l 


f—> M 


o 


u 












to 
































UJ 


c 




c 




O 


o 


C r> 


















to 


c 




o 




o 


o 


C5 












to 




1— 




• 




■ 




■ 


• 














»— 




u 


cj 


*-» 




rvi 






cy 


*-4 












co 


3 


EL 
















cr 












C 


X 




o 














to 








X 


UJ 


i— 


M 


• 


*— k 














►-« 








<T 


e 


Ql 


X 


> 


: 






















U_ 


c 


3 


fe—i 


• 


<* 






















X 


X 




X 


O 


> 
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VAR034 TREND GOVT APPLIED 
CODE 



rv; rc o c >c 

N IP O O K 
N C C 

tT* K» 



O" 



* 

* 

* 

* 

* 

* 

-* 

* 

* 

* 

■* 

-K 

* 

* 

•* 

-It 

♦ 



<x 

r-n 















•* 












CC 


> 


03 


T~ 




o 














•It 








o 




or 


Ui 


Uj 




• 


X 














•It 








=3 




Li* 


C 


X. 


X 


u 


•— t 


*-> 












-* 


















•— 1 




03 






<r-* 








•It 












Cc 


G 


Uj, 


X 


LT. 


03 






in 








•If 




•It 








1— 


•— 




•a 


cr 


►— 














•It 




•It 








03 


03 


03 


X 


• 


X 














•It 




•It 
































-It 




•It 
































-If. 




•If 
































•It 




* 






















LJ 


w> 


Lli 


^3 




* 




•If 


*— 


G 
















•* 


03 




in 






-If 




■K 




IVi 
















•r 


< 


-If 






£3 


•IT 


lu 


•It 






>- 


-- 


O 


K3 


c 




tr» 


♦ 


Ui 


-If 


Ui 




Ui 


•It 


—1 


•* 






tJ 




G 


in 


c 


U3 


•*— 


•if 


CC 


•It 


Cl 




M 


•If 


CL 


■It 






X 


ru 


C 


in 


G 


X 




•ic 


u 


•It 


u 




QC 


-* 


<1 


•H 






Ui 














•ic 


X 


•If 


Ui 


w 


■<f 


■It 


1 — 


-It 








m 


cr 


« 


•— 


in 




*r 


M 


•If 


c 




> 


* 


03 


•* 






G 










K> 




■it 




* 




* 




* 




•If 






U- 














■it 




•If 




•If 




* 




•It 






cc 














•r 




►-» -It 


•— * 


1— -It 




-It 




•It 


M ^ 


G 


U. 












03 




































Ui 


o 




o 




O 




o 




o ^ 


















cr 


o 




o 




o 




o 




C3 












03 




I- 




• 




• 




• 




« 




X 












*— ♦ 


X 


C 


<_> 


•«— » 




ru 








a- 




M 












03 


3 


Q- 




















03 












e 


X 




G 


















03 








X 


UJ 


•— 


•— * 


• 




















»— < 








<t 


C 


or 


X 


> 




















X 








Ui 


c 


3 


M 


• 


< 


























X 


X 




X 


O 


> 



o 

o 



o 

<© 



o 

>o 



UJ 

u 

X 2 
< < Ui 
H H c 

c a Z X 
Ui < < 3 

x :> a: 03 



cC DO ^ c >c 
c vC a o 5 
w o o o 

cr*q y) 



ru 



U. 

40 

<x. 



o: 
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VAR035 TREND GDVT DEVELOPMENT 
CODE 























Oj c 


G 
























o 


CO G 


G 


CJ 
























tr c 


G 


rO 






















• 


• • 


• 


• 






















to 


^*■4 




to 






















































to 


















•—» 


G 




























G 






























































Ui 




Q 
























u 




1- 






















2 


2 


























-<x 


< Ui 


























Hi 


M O 


























G 


a: x 


X 
























U_ 


<t **. 


Z) 


















*-* 


G 




z 


> a 


CO 




















CC 






















-If 




























k 




























-It 




























-If 




























-H 










>c 


co to 


o 


•wH 


o 










-It 












un >o 


o 


CO 


ro 










-It 












oj cc 


c 


CC 












-It 






O 






















-n 






-C 






» 




OJ 












* - 




























k 


















CO 










k 


















Ui 










-If 


















co 










k 


















-«s 










k 


O 
















tJ 










-it 


tO 


f • 








to 




• 




lO 








k 










rt 


> to 


X 


*—e 


G 


rvj 








-It 






G 




OL 


Ui Ui 




• 












k 






O' 




Ui 


G 2 


X 


u 


*-» 










-It 












2 


►- 




CO 










k 


*— «* 








c 


G UJ 


X 


in 


to 










-k 










1— 


i- ¥ 




cr 


•—» 










-k 


k 








CO 


to to 


^_ 


• 


X 










-k 


k 


















k 




— > 


*— * 


-k 


k 


















k 






r- 


-k 


k 


















k 








-k 


k 


















* 


LU 


u_ 




-k 


k 


K— 


G 














* 


CO 


to 




k 


k 




rvj 














k 


-< 


W < 


g 


-k UJ 


k 






>- 




G 


o 


to 




-it 


JLi 


Ui 


U> 


-k _j 


k 






U 


«— 1 


o vr 


o 


ro 




♦ 


ql 


* a: 


v> »— 


-k a: 


k 






2 




G O 


o 






♦ 


o 


-It u 


QC 


-k <r 


k 






UJ 












* 


2: 


-If UJ 


-It C 


k i— 


k 








to 


O — ♦ 




G 




-it 




* o 


k > 


k to 


k 






o 




1 




^T 




-it 




-If 


k 


k 


k 






UJ 












* 




-It 


-It 


k 


k 






Et 












k 


•— » 


1-4 + l-H Hi 


-It H-i 


>— • k »—> 


»—• k 


►H *— 


G 


U- 










CO 




























Ui 


C 




G 


o 


O 


c ^ 
















CO 


G 




O 


G 


G 


O 










to 




t— 


<r 


• 




• 


* 


• 


ZL 










t-H 


X 


tJ 


u 


<*- 




rvi 




O' 


#—» 










CO 




Q- 














in 










c 


X 




G 












in 








2 


Ui #~ 


*-« 


• 


*— i 












»-» 








<r 


c a: 


zz 


> 


_i 












21 








Ui 


C 2 


M 


• 






















X 


X ^ 


X 


O 


> 
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VAR036 TREND CfWRClAL BASIC 
CODE 



-c e o oj 

>C G O CC 
CC ^ G O SC 



CSJ 

cc 



M G 

• c 






m <V m 
cc 
rr 



u #- 























< 


-«r 


UJ 


















» 










*— » 


HH 


cr 




























CL 


cr 


2: 


X 
















-n 










U. 


<* 




















* 






G 




X. 


> 


tr 


cc 
















-It 






cc 


























-It 
































-It 
































-It 
































-It 
































* 
































-* 










ru 


o 


in 


o 


O 


CT 












-It 










o 


ac 


in 


G 




m 












-K 










— » 


o 


m 


G 
















-It 




M 


o 


























-It 
















^T 


tn 














-It 














K 


















■It 




















CO 












-It 


4— « 


















UJ 












-It 




















cc 












-It 


m 


















< 












-r 




















CJ 












-it 














cc 




• 














-it 










a. 


> 


cc 


X 


M 


cr 












•it 




fc— f 


c 




cr 


Ll 


U- 


, 


■ 














* 


V-/ 








Ui 


C 


2 


X 


u 


H-i 












-it 














-3. 


#-* 




cc 












-it 


* 








c. 


£1 


Ll. 


X 


LT 


CC 


«r— 




4-1. 






-it 


-K 








»— 


1- 


r 


< 


O 


H- f 






m 






-it 


-It 








cc 


cc 


cc 




• 


X 


•-» 




<— 






-it 


it 






























* 


-It 




























4—1 


-it 


-r 




























LT 


-* 


-it 






















(i. 




bJ 




-it 


-it 
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VAR037 TREND COHMFRCIM. APPLIED 
CODE 
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VAR03R PRIMARY EVALUATION CRITERION 
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VALID CASES t ^ ? MISSING CASES 



PE AH IMPI.IT DATA 

20131275230 123555230133 t 1 l 1 11 l 110000007 003 
55t3l275U5t335558 8555342lUlin000n00700O 
123221 75532023555 20i 13303111 1 1 1 toooo 400006 
5503? 3005520 2 3555231 23211 1 U U 11 oouaoa^O 1 3 
1 1 02 22121 1513 2555 2 0 1 3321 1 1 1 1 2 1 1 1 22222260 1 6 
2301 1175125213555783215321 1221 11 OU000701P 
5511 till 125123155231 I33t2ll 1 1 1 1100 0 0221020 
560 21. 0703OO13O555201. 1331 1 1 1 I 1 1 1 100 0 0000021 
3322127513012355577323321 1 11 1 1 1 1 0000007022 

023221551301221 118152321231115550001117020 
350222751 63OO30OO?3523301 1 12301 10000005028 
2S22 11 75 1250 12555233232 11 11111110000001036 
2012311 112553303222223011 t 305 1 1 222323 3 7 0 oo 
13 03111112512 2555 221 23311111111100000 oo OOP 
2221 11751625125551332123331 l l 1 1 10000005051 
000031 0135 331 1555731 1301 1 it l 1 1 1 io 000 001052 
100121 75 Il51321552102l3ltt1111ltoaooaoao 56 
1022217512512255561 1 tl0133llllll000000005P 
120222751301020553310101321111110000007066 
1 31 221 1 0 1 1 50330 55231 21 31 1 100000 1 0000021 068 
2031 1 135165533555 1.01251 l 1 1000000000001706P 
1301217! 3621235553722331 1 1 1 1 1 1 1 10400001071 
22301 1751635. 33555 025235221 121 1 1 1 1 1 10007076 
260121 001531235558101 t 11111101113330137080 
113221651300301116850055555555550000005080 
1300 U751150000551 73210H11111H0000007091 
U1. 32161 163H1055272O1O OH. Itllll000000t093 
13 U1 17511 551 25552301 1 1 1 125551 1 1 0000 0 07097 
13031 1 7516101 10 558 31 0031 1 1 1 1 111 l'lOOOOoOO 98 
232322121 301 125552822131 1 1 1 1 1 1 1 1 000000 1 100 
2311. 1 1021001205557132321 i 11 1 1 1 11 0000005101 
3312201 OOP01235S521 2133221 1 1 12222223331 1 02 
310225101231 1 1 1 1 11 732 32222 1 33 1 1 2 2 1 l 0006 1 0 3 
561222021 251235552312331 11 1 1 1 1 1 10001 1 1 1 1 n 
201221701 6332355523353321 1 l 1 1 1 1 1 0 0 t o 1 26 1 1 2 
132311751530221337812130331311 110110025113 
1 33221 6525313O50O0000A 2555 tilt 1 11102200 no 

1611 1 1751 1510355521 l?l3tl 1 1 1 1 1 1 10010 007121 
501 2i5 1233 o?3o 532232103112 0003330000006122 

00 0 311 05 1130 2 30 50 7332220220120 1 10010001123 
5510 1 161 1 151 135551.7 0030 121 1 111 1 1 1 111 1 10128 
3520U 701 151235552301 131 1 1 1 1 1 1 1 10000225129 
5303317512512255533323511 11111112212017131 

0001 12151 62133555231 131 1. 1 1 1 1 1 1 1 1 00000 4 1 132 
20 122511 1351 12555731 1 321 It 1 1 1 1 1 10440046 133 
222221751651235552312131 111111500444101130 
20 121 1751 201325557332331 U 1 1 1 1 1 1 0000007136 
2511117515312355573311311111111100 OOOOOtOl 
222011752001235557211121111111111110006102 
233321 751531325556 ii? i ?321 1111210400105145 
05022275 I53l235552832l3t1 111 1 1 1 10000007153 
230322751533235552321301110010000010001155 
23021 1751151331 1^711 1 331 1 1 1 121 1 10000001156 
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544212751431 13 555211 1. 335 1 1 555 i. U 222222 1 1 5 7 
564221121251225557311 531 l 11111110444447161 
122121750 4 2333555731112211111111444444 1. U3 
111311 751251235557311334H 1111114444441164 
361. t 1.1 751 15133555214 1 321 1 1 1 1 1 1 1 1 3333334 166 
241 2221.21 1513323576 32131 1 l 1 1 1 1 1.1 1 1 12417167 
334111451630230552352331111111110444445172 
25121275155433533231 1 321 121 341 It 1 1444261. 73 
204211751 151335552322331 111111112112111175 
2321 1.1751 341 135556751331 1 1000000 0000006177 
252121751 34 13355566? J33U 11111110000007175 
21 22 2375 153130 555 2 34 1331121 111112224446175 
1321 1 1001200 125557312131 1 1 1 1 1 1 1 14444041 1 61 
2524 1 175134 12255523113311 nt11tl«llAll7l*12 
142411001111111115304315550000000000006102 
033121011531335552712121111111110000441106 
334411751255115551112231121111110444444107 
232231751203225447342332111111 t 1 2202426203 
14322141 1. 610220 55 731?1301111 1 1 1 10000000205 
123421751250231 1 11652525225555541 1 t 1 117213 
132121752442325557332131111111114444441216 
123131251240005557242122223333330000006225 
033331151150030052143330000000010000024227 
113211411341125552312331131111114444446234 

342232751 3412355573123321 11 1 1 1 1 1 3333317237 
22 111275 1330 33 1 5573153231111111144 44U4424P 
234U37515312355523123351 1 1 1 1 111444 4 446257 
1231 1 15016301 1055231 1 1 3t t 1 1 l 1 1 1 14444445273 

3622207524332355571 1.2331 1 1 1 1 10004440001274 

4521 1. 1751 2413255521 12331 1 1 1 1 1 1 1 14414447276 
5541131314412355567U3211U111 1 14444441261 
232 t t 55534 3 134555231 4331 121111110000001260 
2122101135332355571121211 10000000000007200 
242123144431335553414331 121 1 1 1 l 1444444 1 . 300 
24221 1 4 1 1 l 4033055261 2331 1 1 1 1 1 1 1 14444447305 
221221751250330551132110221111110000114307 
2423227512403355023313311111 ' 1114444441306 
231 122701 13133555231 1331 1 l 121 l t 14444441300 
332123751 6313315528141 U U 1 11! 114444441310 
351212121 34432541231. 2321 1 1 125U 1 2442416311 
15240475462323555331 I32lllll111l22?11173l5 
121111151651111552212332342211141111447316 
2521217545353255523413311 1 1 1 i 1 1 1 1 444 1 l 5320 
134121751 631335557312331 111111114444447323 
143321 7 5 161 122550725133121 11 11114444447325 
3322 3 l 0O1635U 555 i l 1 1. 131 1 111 1 1232221 1 1 1 326 
224121751251235556344231 121 1 11114444445327 
252211 12125 52 35 552112331111111114444 4 44328 
123 1.224 01 25 1220552311321111111114111116337 
242211750250210552312130000000000000007343 
222132111251225552311335211111114442141340 
36423275353523554231 1 33 1 1 1 1 2222 11 1 1 1 1 1 735 1 
244221701343235506652131111111114444447350 
16121 1752432235552655331 1 1 1 1 11. 1 1 4444447360 
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03342222225 t 4355561 4 2 325 ^5 i l 31 U 44 1 4444375 
1222 227O123a?l345233?13lU5U0ll111l2<»ll 377 
231212751 3<il 3454 5731 1 121 1 1 1 22 U 222333 1 63*0 
aa 1 1 12751 1*13255563523321211 li 1144344363*7 
122221701 151 1 155523 42131 1 1 1 1 U 1 14 44 til HI 3 
2321 23 751 fe 1^2 155528 1.2331 1 1 1 1 1 1 1 1 4444447423 
2322211111511 2555232 113111111 lll«a«aauia23 
231. 1 12121 1551 2555 1« 121211111 11 ! 1222222 1442 
3 a i_ 131751 2^1^255521 1.2331 1 1 1 1 1 11 10000004451 
03342111261. 1230001212305 *5000000000 0 0014 65 
361 i 32701 a ua 23555731 1 31 1 1 1 1 1 1 1 1 14444441466 
35a 1 12751531 1355573323311 11 1 1 1 1 133343474 7 2 
2534237556152155572513311 1 U 1 1 11 0000007478 
3311127013413322221313211 1 243 1 1 1 4 l i i U 6435 
221121311151325552312431111111114444411462 
121322^51151 121 15333^31 001 1111114444441467 
334415701250120551315330230010220010016501 
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